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Nursing  homes  play  an  important  role  in  the  health  care  system  of  the 
United  States.  More  than  40  percent  of  elderly  Americans  will  use  a 
nursing  home  at  some  time  in  their  lives.  Such  facilities  provide  skilled 
nursing,  therapy,  or  supportive  care  to  older  individuals  who  do  not  need 
the  intensive  medical  care  provided  by  hospitals,  but  for  whom  receiving 
care  at  home  is  not  feasible.  Under  the  Medicare  and  Medicaid  programs, 
nursing  homes  were  expected  to  receive  $58  billion  in  2001,  with  a  federal 
share  of  approximately  $38  billion.  Nursing  homes  that  participate  in  these 
programs  are  required  to  periodically  assess  the  care  needs  of  residents  m 
order  to  develop  an  appropriate  plan  of  care.  Such  resident  assessment 
data  are  known  as  the  minimum  data  set  (MDS).^  The  federal  government 
contracts  with  states  to  periodically  inspect  or  survey  nursing  homes,  and 
state  surveyors  use  MDS  data  to  help  assess  the  quality  of  resident  care.^ 
Medicare  and  some  state  Medicaid  programs  also  use  MDS  data  to  adjust 
nursing  home  payments  to  account  for  variation  in  resident  care  needs. 


^The  Omnibus  Budget  Reconciliation  Act  of  1987  required  the  Secretary  of  Health  and 
Human  Services  to  specify  a  minimum  data  set  of  core  elements  to  use  in  conducting 
comprehensive  assessments  of  patient  conditions  and  care  needs.  See  42  U.S.C.  §  1395i-3; 
42  U.S.C.  §  1396r.  By  mid-1991,  the  requirement  to  assess  and  plan  for  resident  care  had 
been  implemented  in  all  nursing  homes  that  serve  Medicare  and  Medicaid  beneficiaries. 
MDS  data  are  collected  for  all  residents  in  these  facilities,  including  Medicare,  Medicaid, 
and  private  pay  patients. 

^The  federal  government  has  responsibility  for  establishing  requirements  that  nursing 
homes  must  meet  to  participate  in  publicly  funded  programs.  Every  nursing  home  that 
receives  Medicare  or  Medicaid  funding  must  undergo  a  standard  survey  conducted  on 
average  every  12  months  and  no  less  than  once  every  15  months.  Under  its  contracts  with 
states,  the  federal  government  funds  100  percent  of  costs  associated  with  certifying  that 
nursing  homes  meet  Medicare  requirements  and  75  percent  of  the  costs  associated  with 
Medicaid  standards. 
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Thus,  the  accuracy  of  MDS  data  has  implications  for  the  identification  of 
quality  problems  and  the  level  of  nursing  home  payments. 

MDS  accuracy  is  one  of  many  areas  that  state  surveyors  are  expected  to 
examine  during  periodic  nursing  home  surveys.  Federal  guidance  for  state 
surveyors  regarding  the  accuracy  of  MDS  assessments  focuses  on  whether 
appropriate  personnel  completed  or  coordinated  the  assessments  and 
whether  there  are  any  indications  that  the  assessments  were  falsified.  This 
guidance  also  instructs  surveyors  to  conduct  a  check  of  specific  MDS 
items  to  ensure  that  the  resident’s  condition  is  appropriately 
characterized.  Concerns  exist,  however,  that  state  surveyors  already  have 
too  many  tasks  and  that,  as  a  result,  the  survey  process  may  not 
adequately  address  MDS  accuracy.  In  addition,  our  prior  work  on  nursing 
home  quality  issues  has  identified  weaknesses  in  the  survey  process  that 
raise  questions  about  the  thoroughness  and  consistency  of  state  surveys.® 

In  response  to  your  request,  we  assessed  (1)  how  states  monitor  the 
accuracy  of  MDS  data  compiled  by  nursing  homes  through  review 
programs  separate  firom  their  standard  nursing  home  survey  process, 

(2)  how  states  attempt  to  improve  the  data’s  accuracy  where  there  are 
indications  of  problems,  and  (3)  how  the  federal  government  ensures  the 
accuracy  of  MDS  data.  We  surveyed  the  50  states  and  the  District  of 
Columbia  to  determine  whether  states  had  a  separate  MDS  review 
program — distinct  from  any  MDS  oversight  that  might  occur  during  the 
periodic  nursing  home  surveys  performed  by  all  states.  We  then  conducted 
structured  interviews  with  officials  in  10  of  the  11  states  that  indicated 
they  had  separate  MDS  review  programs.^  We  also  interviewed  staff  from 
the  Centers  for  Medicare  and  Medicaid  Services  (CMS),  an  agency  within 
the  Department  of  Health  and  Human  Services  (HHS)  that  manages  the 
Medicare  and  Medicaid  programs,  who  were  responsible  for  developing 


®See  Nursing  Homes:  Sustained  Efforts  Are  Essential  to  Realize  Potential  of  the  Quality 
Initiatives  (GAO/HEHS-00497,  Sept.  28, 2000). 

^These  10  states  are  Iowa,  Indiana,  Maine,  Mississippi,  Ohio,  Pennsylvania,  South  Dakota, 
Vermont,  Washington,  and  West  Virginia.  Due  to  the  newness  of  Virginia’s  MDS  review 
program  (implemented  in  April  2001),  we  focused  on  the  experience  of  the  10  states  with 
longer  standing  programs.  In  addition,  about  one-third  of  the  states  without  separate  MDS 
review  programs  volunteered  additional  information  regarding  the  ways  in  which  the 
accuracy  of  MDS  data  may  be  addressed  through  the  nursing  home  survey  process  or 
training  programs  offered  by  the  state. 
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the  agency’s  MDS  review  program.'  In  addition,  we  reviewed  regulations, 
literature,  and  other  documents  relating  to  MDS  data.  We  performed  our 
work  from  December  2000  through  January  2002  in  accordance  with 
generally  accepted  government  auditing  standards. 


Results  in  Brief 


Eleven  states  have  established  separate  MDS  review  programs,  apart  from 
their  standard  nursing  home  survey  process,  to  monitor  the  accuracy  of 
resident  assessment  data  compiled  by  nursing  homes.  An  additional  seven 
states  reported  that  they  plan  to  do  so.  According  to  officials  in  the  10 
states  with  MDS  accuracy  review  programs  in  operation  as  of  January 
2001,  these  programs  were  established  primarily  because  of  the  important 
role  played  by  MDS  data  in  setting  Medicaid  payments  and  identifying 
quality  of  care  problems.  While  routine  nursing  home  surveys  provide  an 
opportunity  to  examine  the  accuracy  of  MDS  data,  officials  in  some  of  the 
10  states  with  separate  MDS  review  programs  told  us  that  surveyors  do  not 
have  sufficient  time  to  focus  on  the  data’s  accuracy  because  of  other 
survey  tasks.  To  assess  the  accuracy  of  the  MDS  data,  9  of  the  10  states 
conduct  periodic  on-site  reviews  in  all  or  a  significant  portion  of  their 
nursing  homes.  These  reviews  include  checking  a  sample  of  a  home’s  MDS 
assessments  and  determining  whether  the  basis  for  the  assessments  is 
adequately  documented  in  residents’  medical  records.  In  addition,  these 
reviews  often  include  interviews  of  nursing  home  personnel  familiar  with 
residents  and  observations  of  the  residents  themselves.  Such 
corroborating  evidence  provides  reviewers  increased  assxirance  that  an 
MDS  assessment  accurately  reflects  the  resident’s  condition.  States  with 
on-site  review  programs  reported  that  the  discrepancies  they  identified 
between  MDS  assessments  and  the  supporting  documentation,  also  called 
“MDS  errors,”  typically  resulted  from  differences  in  clinical  interpretation 
or  mistakes,  such  as  a  misxmderstanding  of  MDS  definitions.  Two  of  the  10 
states  were  able  to  teU  us  the  amount  of  the  recoupments  they  obtained 
from  nursing  homes  due  to  Medicaid  overpayments  based  on  inaccurate 
MDS  assessments.  For  example.  West  Virginia  received  $1  million  from 
one  nursing  home  relating  to  MDS  errors  associated  with  physical  therapy 
services. 


^On  June  14,  2001,  the  Secretaiy  of  HHS  changed  the  name  of  the  Health  Care  Financing 
Administration  (HCFA)  to  CMS.  In  this  report,  we  wiU  continue  to  refer  to  HCFA  where  our 
findings  apply  to  the  organizational  structure  and  operations  associated  with  that  name. 
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States  with  separate  MDS  review  programs  identified  a  variety  of 
approaches  to  improving  MDS  accuracy.  State  officials  highlighted  the  on¬ 
site  review  process  itself  and  provider  education  activities  as  their  primary 
approaches.  On-site  reviews  heighten  facility  staff  awareness  of  the 
importance  of  MDS  data  and  can  lead  to  the  correction  of  practices  that 
contribute  to  MDS  errors.  Some  officials  said  that  on-site  reviews  provide 
a  valuable  opportunity  for  informal  training  and  coaching  staff  about 
completing  and  documenting  MDS  assessments,  which  is  important  given 
the  types  of  MDS  errors  found  and  the  high  staff  turnover  in  nursing 
homes.  Identifying  areas  of  confusion  by  nursing  home  staff  during  on-site 
MDS  reviews  is  also  useful  in  guiding  the  focus  of  formal  training  sessions 
conducted  outside  of  the  nursing  home.  State  officials  reported  that  they 
also  have  one  or  more  remedies  at  their  disposal  to  help  improve 
accuracy,  such  as  requiring  nursing  homes  to  prepare  a  corrective  action 
plan  or  imposing  financial  penalties  on  nursing  homes  when  serious  or 
extensive  errors  in  MDS  data  are  found.  Indiana,  for  example,  requires 
facilities  to  submit  a  corrective  action  plan  detailing  how  the  facility  will 
address  errors  identified  during  an  on-site  review.  In  addition,  Maine  has 
collected  approximately  $390,000  in  financial  penalties  since  late  1995 
from  facilities  with  MDS  errors.  Finally,  officials  from  five  states  told  us 
that  their  MDS  review  efforts  have  resulted  in  a  notable  decrease  in  MDS 
errors  across  all  facilities.  For  example,  the  average  percentage  of 
assessments  with  MDS  errors  that  resulted  in  a  payment  change  since 
initiation  of  their  separate  review  programs  has  decreased  from  about  85 
percent  to  10  percent  of  assessments  in  South  Dakota  and  from  75  percent 
to  30  percent  of  assessments  in  Indiana. 

Following  the  1998  implementation  of  Medicare's  MDS-based  payment 
system,  the  Health  Care  Financing  Administration  (HCFA)  began  building 
the  foundation  for  its  own  separate  review  program — distinct  from  state 
efforts — ^intended  to  ensure  the  accuracy  of  MDS  data  for  all  nursing  home 
residents.  In  the  course  of  developing  and  testing  various  accuracy  review 
approaches,  an  agency  contractor  found  widespread  MDS  errors  that 
resulted  in  a  change  in  the  Medicare  payment  level  for  two-thirds  of  the 
resident  assessments  sampled.  Its  on-site  visits  proved  to  be  a  very 
effective  method  of  assessing  accuracy.  As  a  result,  the  contractor 
recommended  that  any  MDS  reviews  involve  on-site  visits,  at  least  for  the 
first  few  years  of  any  national  review  program,  along  with  certain  off-site 
analysis  to  help  target  homes  and  areas  for  review.  In  September  2001, 
CMS  awarded  a  new  contract  to  establish  a  national  MDS  accuracy  review 
program.  As  currently  planned,  CMS’  MDS  review  activities  are  projected 
to  involve  roughly  1  percent  of  the  estimated  14.7  million  MDS 
assessments  expected  to  be  completed  in  2001,  with  on-site  reviews  in 
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fewer  than  200  of  the  nation’s  17,000  nursing  homes  each  year.  In  contrast, 
states  that  conduct  separate  MDS  reviews  typically  examine  from  10  to  40 
percent  of  assessments  completed  in  all  or  a  significant  portion  of  their 
nursing  homes.  The  CMS  contractor  is  required  to  coordinate  its  activities 
with  ongoing  state  and  federal  efforts.  For  example,  to  avoid  unnecessary 
overlap,  the  contractor  is  instructed  to  coordinate  with  states  regarding 
the  selection  of  facilities  and  the  timing  of  visits.  However,  the  contractor 
is  not  specifically  tasked  with  assessing  the  adequacy  of  each  state’s  MDS 
accuracy  activities.  While  CMS’  approach  may  yield  some  broad  sense  of 
the  accuracy  of  MDS  assessments  on  an  aggregate  level,  it  appears  to  be 
insufficient  to  provide  confidence  about  the  accuracy  of  MDS  assessments 
in  the  vast  bulk  of  nursing  homes  nationwide. 

Given  the  substantial  level  of  effort  and  resources  already  invested  at  the 
state  and  federal  levels  to  oversee  nursing  home  quality  of  care,  including 
periodic  inspections  at  each  home  nationwide,  we  believe  that  CMS  should 
reorient  its  MDS  accuracy  program  so  that  it  complements  and  leverages 
existing  state  review  activities  and  its  own  established  nursing  home 
oversight  efforts.  Therefore,  we  are  making  recommendations  to  the 
administrator  of  CMS  that  include  determining  the  adequacy  of  each 
state’s  efforts  to  ensure  MDS  accuracy  and  providing  additional  guidance 
and  technical  assistance  to  individual  states  as  needed;  routinely 
monitoring  state  review  activities  and  progress  as  part  of  CMS’  own 
ongoing  federal  oversight  of  nursing  home  quality;  and  ensuring  that  states 
and  nursing  homes  have  sufficient  documentation  to  support  the  full  MDS 
assessment. 

In  commenting  on  a  draft  of  this  report,  CMS  agreed  with  the  importance 
of  assessing  and  monitoring  the  adequacy  of  state  MDS  accuracy  efforts. 
CMS  recognized  that  the  MDS  impacts  reimbursement  and  care  planning 
and  that  it  is  essential  that  the  assessment  data  reflect  the  resident’s  health 
status  so  that  the  resident  may  receive  the  appropriate  quality  care  and 
that  providers  are  appropriately  reimbursed.  While  CMS’  comments 
suggested  that  its  current  efforts  may  be  sufficient  to  assess  and  improve 
state  performance,  we  do  not  believe  they  will  result  in  the  systematic 
assessment  and  monitoring  of  each  state’s  MDS  accuracy  that  we 
recommended.  CMS  did  not  agree  with  our  recommendation  on  the  need 
for  sufficient  documentation  to  support  the  full  MDS  assessment, 
expressing  concern  about  potential  duplicative  effort  and  unnecessary 
burden  for  nursing  homes.  In  our  view,  documentation  need  not  be 
duplicative,  but  demonstrative  that  the  higher-level  summary  judgment 
about  a  resident’s  condition  and  needs  entered  on  the  MDS  can  be 
independently  validated.  Given  the  importance  of  MDS  data  in  ac^usting 
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nursing  home  payments  and  guiding  resident  care,  ensuring  their  integrity 
is  critical  to  achieving  their  intended  piuposes. 


Background 


The  nation’s  17,000  nursing  homes  play  an  essential  role  in  our  health  care 
system,  providing  services  to  L6  million  elderly  and  disabled  persons  who 
are  temporarily  or  permanently  unable  to  care  for  themselves  but  who  do 
not  require  the  level  of  care  furnished  in  an  acute  care  hospital.  Depending 
on  the  identified  needs  of  each  resident,  as  determined  through  MDS 
assessments,  nursing  homes  provide  a  variety  of  services,  including 
nursing  and  custodial  care,  physical,  occupational,  and  speech  therapy, 
and  medical  social  services.®  The  majority  of  nursing  home  residents  have 
their  care  paid  for  by  Medicaid,  a  joint  federal-state  program  for  certain 
low-income  individuals.  Almost  all  nursing  homes  serve  Medicaid 
residents,  while  more  than  14,000  nursing  homes  are  also  Medicare- 
certified.  Medicare,  the  federal  health  care  program  for  elderly  and 
disabled  Americans,  pays  for  posthospital  nursing  home  stays  if  a 
beneficiary  needs  skilled  nursing  or  rehabilitative  services.^  Medicare- 
covered  skilled  nursing  home  days  account  for  approximately  9  percent  of 
total  nursing  home  days.  Medicare  beneficiaries  tend  to  have  shorter 
nursing  home  stays  and  receive  more  rehabilitation  services  than 
individuals  covered  by  Medicaid. 


MDS  Used  to  Assess  Since  1991,  nursing  homes  have  been  required  to  develop  a  plan  of  care  for 

Nursing  Home  Residents  resident  based  on  the  periodic  collection  of  MDS  data.  The  MDS 

contains  individual  assessment  items  covering  17  areas,  such  as  mood  and 
behavior,  physical  functioning,  and  skin  conditions.  MDS  assessments  of 
each  resident  are  conducted  in  the  first  14  days  after  admission  and  are 


®For  patients  with  an  advanced  illness,  medical  social  services  generally  help  the  patient 
and  family  cope  with  the  logistics  of  daily  life,  including  financial  and  legal  planning  and 
mobilizing  community  resources  that  may  be  available  to  the  patient.  Such  sendees  may 
also  include  counseling  the  patient  and  family  to  address  emotions  and  other  issues  related 
to  the  advanced  illness. 

^To  qualify,  a  Medicare  beneficiary  must  require  daily  skilled  nursing  or  rehabilitative 
therapy  services,  generally  within  30  days  of  a  hospital  stay  of  at  least  3  days  in  length,  and 
must  be  admitted  to  the  nursing  home  for  a  condition  related  to  the  hospitalization. 
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used  to  develop  a  care  plan.®  A  range  of  professionals,  including  nurses, 
attending  physicians,  social  workers,  activities  professionals,  and 
occupational,  speech,  and  physical  therapists,  complete  designated  parts 
of  the  MDS.^  Assessing  a  resident’s  condition  in  certain  areas  requires 
observation,  often  over  a  period  of  days.  For  example,  nursing  staff  must 
assess  the  degree  of  resident  assistance  needed  during  the  previous  7 
days — ^none,  supervised,  limited,  extensive,  or  total  dependence — ^to  carry 
out  the  activities  of  daily  living  (ADL),  such  as  using  a  toilet,  eating,  or 
dressing.  To  obtain  this  information,  staff  completing  the  MDS 
assessments  are  required  to  communicate  with  direct  care  staff,  such  as 
nursing  assistants  or  activities  aides,  who  have  worked  with  the  resident 
over  different  time  periods.  These  staff  have  first-hand  knowledge  of  the 
resident  and  wiU  often  be  the  primary  and  most  reliable  source  of 
information  regarding  resident  performance  of  different  activities.  While  a 
registered  nurse  is  required  to  verify  that  the  MDS  assessment  is  complete, 
each  professional  staff  member  who  contributed  to  the  assessment  must 
sign  and  attest  to  the  accuracy  of  his  or  her  portion  of  the  assessment. 


MDS  Used  in  Quality 
Oversight  and  as  Basis  for 
Payments 


MDS  data  are  also  submitted  by  nursing  homes  to  states  and  CMS  for  use 
in  the  nursing  home  survey  process  and  to  serve  as  the  basis  for  ac^justing 
payments.  CMS  contracts  with  states  to  periodically  survey  nursing  homes 
to  review  the  quality  of  care  and  assure  that  the  services  delivered  meet 
the  residents’  assessed  needs.  In  fiscal  year  2001,  the  federal  government 


®MDS  assessments  are  conducted  for  all  nursing  home  residents  within  14  days  of 
admission  and  at  quarterly  and  yearly  intervals  unless  there  is  a  significant  change  in 
condition.  Accommodating  their  shorter  nursing  home  stays,  Medicare  beneficiaries  in  a 
Medicare-covered  stay  are  assessed  on  or  before  the  5th,  14th,  and  30th  day  of  their  stays 
and  every  30  days  thereafter. 

^  a  recent  study,  the  HHS  Office  of  Inspector  General  (OIG)  reported  that  almost  all  of 
the  facilities  in  its  study  had  a  position  of  MDS  coordinator.  Eighty-one  percent  were 
registered  nurses,  and  the  remainder  were  either  licensed  practical  nurses,  licensed 
vocational  nurses,  or  social  workers.  See  HHS  OIG,  Nursing  Home  Resident  Assessment: 
Quality  of  Care^  OEI-02-99-00040  (Washington,  D.C.:  HHS,  Dec.  2000). 
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spent  about  $278  million  on  the  nursing  home  survey  process.^®  Effective 
July  1999,  the  agency  instructed  states  to  begin  using  quality  indicators 
derived  from  MBS  data  to  review  the  care  provided  to  a  nursing  home’s 
residents  before  state  surveyors  actually  visit  the  home  to  conduct  a 
survey.”  Quality  indicators  are  essentially  numeric  warning  signs  of 
potential  care  problems,  such  as  greater-than-expected  instances  of  weight 
loss,  dehydration,  or  pressure  sores  among  a  nursing  home’s  residents. 
They  are  used  to  rank  a  facility  in  24  areas  compared  with  other  nursing 
homes  in  a  state.  In  addition,  by  using  the  quality  indicators  before  the  on¬ 
site  visit  to  select  a  preliminary  sample  of  residents  to  review,  surveyors 
should  be  better  prepared  to  identify  potential  care  problems. 

In  addition  to  quality  oversight,  some  state  Medicaid  programs  and 
Medicare  use  MBS  data  to  adjust  nursing  home  payments  to  reflect  the 
expected  resource  needs  of  their  residents.  Such  payment  systems  are 
commonly  known  as  “case-mix”  reimbursement  systems.  Because  not  all 
residents  require  the  same  amount  of  care,  the  rate  paid  for  each  resident 
is  adjusted  using  a  classification  system  that  groups  residents  based  on 
their  expected  costs  of  care.  Facilities  use  MBS  data  to  assign  residents  to 
case-mix  categories  or  groups  that  are  defined  according  to  clinical 
condition,  functional  status,  and  expected  use  of  services.  In  Medicare, 
these  case-mix  groups  are  known  as  resource  utilization  groups.  Each 
case-mix  group  represents  beneficiaries  who  have  similar  nursing  and 
therapy  needs.  As  of  January  2001, 18  states  had  introduced  such  payment 
systems  for  their  Medicaid  programs.^^  As  directed  by  the  Congress,  HCFA 


assess  state  survey  agency  performance  in  fulfilling  contractual  obligations,  CMS  is 
required  by  statute  to  conduct  federal  oversight  surveys  in  at  least  5  percent  of  the  nursing 
homes  in  each  state  within  2  months  of  the  state’s  completion  of  its  survey.  CMS  fulfills  this 
requirement  by  conducting  a  combination  of  (1)  comparative  surveys,  in  which  a  federal 
team  independently  surveys  a  nursing  home  recently  Inspected  by  a  state  in  order  to 
compare  and  contrast  the  results,  and  (2)  observational  surveys  where  federal  surveyors 
accompany  a  state  survey  team  to  a  nursing  home  to  watch  the  conduct  of  the  survey, 
provide  immediate  feedback,  and  later  rate  the  team’s  performance.  Comparative  surveys 
offer  a  more  accurate  picture  of  the  adequacy  of  state  survey  activities  than  do 
observational  surveys,  which  primarily  are  used  to  help  identify  training  needs.  HCFA 
surveyors  found  deficiencies  that  were  more  serious  than  those  identified  by  state 
surveyors  in  about  70  percent  of  the  157  comparative  surveys  they  conducted  between 
October  1998  and  May  2000.  See  GAO/HEHS-00-197,  Sept.  28,  2000. 

^Quality  indicators  were  developed  in  a  HCFA-funded  project  at  the  University  of 
Wisconsin.  See  Center  for  Health  Systems  Research  and  Analysis,  Facility  Guide  for  the 
Nursing  Home  Quality  Indicators  (University  of  Wisconsin-Madison:  Sept.  1999). 

^^We  refer  to  these  states  as  having  “MDS-based  payment  systems.” 
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in  1998  implemented  a  prospective  payment  system  (PPS)  for  skilled 
nursing  facilities  (SNF) — ^nursing  homes  that  are  certified  to  serve 
Medicare  beneficiaries.  The  SNF  PPS  also  uses  MDS  data  to  ac^ust  nursing 
home  payments. 


MDS  Review  Activities  Can  States  and  CMS  use  the  term  “accuracy  reviews”  to  describe  efforts  that 
Be  On-Site  or  Off-Site  help  ensure  MDS  assessments  accurately  reflect  residents’  conditions. 

Review  activities  can  be  performed  on-site — that  is,  at  the  nursing  home 
or  off-site.  On-site  reviews  generally  consist  of  documentation  reviews  to 
determine  whether  the  resident’s  medical  record  supports  the  MDS 
assessment  completed  by  the  facility.''  If  the  MDS  assessment  is  recent, 
the  review  may  also  include  direct  observation  of  the  resident  and 
interviews  with  nursing  home  staff  who  have  recently  evaluated  or  treated 
the  resident. 

While  documentation  reviews  may  also  be  conducted  outside  of  the 
nursing  home,  other  off-site  reviews  of  MDS  data  include  examining  trends 
across  facilities.'^  For  example,  off-site  review  activities  could  involve  the 
examination  of  monthly  reports  showing  the  distribution  of  residents’ 
case-mix  categories  across  different  facilities  in  a  state.  Similarly,  off-site 
reviews  could  also  involve  an  examination  of  particular  MDS  elements, 
such  as  the  distribution  of  ADLs  within  and  across  nursing  homes  to 
identify  aberrant  or  inconsistent  patterns  that  may  indicate  the  need  for 
further  investigation.  Off-site  and  on-site  reviews  may  also  be  combined  as 
a  way  of  leveraging  limited  resources  to  conduct  MDS  accuracy  activities. 


"Each  nursing  home  resident  has  a  medical  record  where  information  about  the  resident  is 
documented.  In  addition  to  the  cuirent  plan  of  care,  examples  of  medical  record 
documentation  include:  (1)  recent  physician  notes,  (2)  results  of  recent  tests,  and 
(3)  documentation  of  services  provided.  Nursing  home  staff  use  this  documentation  to 
complete  each  MDS  assessment.  Maintaining  an  adequate  level  of  documentation  in  the 
medical  record  improves  the  ability  of  staff  to  complete  the  MDS  accurately,  particularly 
for  areas  that  require  observation  over  a  period  of  days.  Some  states  assert  that 
determining  the  degree  of  assistance  that  a  resident  requires  with  ADLs,  such  as  bathing, 
dressing  and  toileting,  requires  repeated  observation  over  several  days,  thus  increasmg  the 
need  for  documentation. 

'^CMS’  current  review  of  SNF  PPS  claims  is  an  example  of  an  off-site  documentation 
review.  CMS  contracts  with  fiscal  intermediaries  to  process  Medicare  claims  and  to 
conduct  reviews  that  use  medical  records  requested  from  nursing  homes  to  ensure  that 
claims  for  Medicare  payments  are  adequately  supported.  For  fiscal  years  2000  and  2001, 
such  contracts  required  fiscal  intermediaries  to  review  0.5  percent  and  1  to  3  percent, 
respectively,  of  total  SNF  PPS  claims. 
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Only  Eleven  States 
Conduct  Separate  On- 
Site  or  Off-Site 
Reviews  of  MDS 
Accuracy 


Eleven  states  conduct  separate  MDS  accuracy  reviews  apart  from  their 
standard  nursing  home  survey  process.  Ten  of  these  states’  reviews  were 
in  operation  as  of  January  2001.  An  additional  7  states  reported  that  they 
intend  to  initiate  similar  accuracy  reviews.^®  All  18  of  these  states  either 
currently  use  an  MDS-based  Medicaid  payment  system  or  plan  to 
implement  such  a  system.  The  remaining  33  states  have  no  plans  to 
implement  separate  MDS  review  programs  and  currently  rely  on  their 
periodic  nursing  home  surveys  for  MDS  oversight.*®  In  all  but  one  of  the 
states  with  separate  MDS  review  programs  operating  as  of  January  2001, 
accuracy  reviews  entail  periodic  on-site  visits  to  nursing  homes.  The 
reviews  focus  on  whether  a  sample  of  MDS  assessments  completed  by  the 
facility  is  supported  by  residents’  medical  records.  If  the  MDS  assessments 
reviewed  are  recent  enough  that  residents  are  still  in  the  facility  and  their 
health  status  has  not  changed,  the  on-site  review  may  also  be 
supplemented  with  interviews  of  nursing  home  staff  familiar  with  the 
residents,  as  weU  as  observations  of  the  residents  themselves,  to  validate 
the  record  review.  About  half  of  these  states  also  conduct  off-site  data 
analyses  in  which  reviewers  look  for  significant  changes  or  outliers,  such 
as  facilities  with  unexplained  large  shifts  in  the  distribution  of  residents 
across  case-mix  categories  over  a  short  period.  Officials  primarily 
attributed  the  errors  found  during  their  on-site  reviews  to  differences  in 
clinical  interpretation  and  mistakes,  such  as  a  misunderstanding  of  MDS 
definitions.  A  few  of  these  states  have  been  able  to  show  some 
recoupments  of  Medicaid  payments  since  the  implementation  of  their  on¬ 
site  review  programs. 


*®In  January  2002,  we  learned  that  one  of  these  states — Kentucky — had  implemented  its 
MDS  review  program  in  October  2001.  Our  analysis,  however,  is  based  on  the  10  programs 
in  operation  as  of  January  2001. 

*®The  District  of  Columbia  is  included  as  one  of  the  33  states  that  has  no  plans  to 
implement  a  separate  MDS  review  program.  In  this  report,  we  generally  refer  to  the  District 
of  Columbia  as  a  state. 
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Most  States  Do  Not  Have 
Separate  MDS  Review 
Programs 


Of  the  50  states  and  the  District  of  Columbia,  only  11  conduct  accuracy 
reviews  of  MDS  data  that  are  separate  from  the  state’s  nursing  home 
survey  process/^  (See  table  1.)  These  11  states  provide  care  to 
approximately  22  percent  of  the  nation’s  nursing  home  residents  and  aU 
but  one  have  an  MDS-based  payment  system  (Virginia  began  conducting 
MDS  accuracy  reviews  in  April  2001  in  anticipation  of  adopting  such  a 
payment  system  in  2002).  Seven  additional  states  plan  to  initiate  separate 
MDS  reviews — ^three  currently  have  an  MDS-based  payment  system  and 
four  are  planning  to  implement  such  a  payment  system.  Officials  in  the  10 
states  with  separate,  longer  standing  MDS  review  programs  said  that  the 
primary  reason  for  implementing  reviews  was  to  ensure  the  accuracy  of 
the  MDS  data  used  in  their  payment  systems.  Several  of  these  states  also 
indicated  that  the  use  of  MDS  data  in  generating  quality  indicators  was 
another  important  consideration.  Vermont  officials,  in  particular, 
emphasized  the  link  to  quality  of  care,  noting  that  the  state  had  created  its 
own  MDS-based  quality  indicators  prior  to  HCFA’s  requirement  to  use 
quality  indicators  in  nursing  home  surveys.  A  state  official  told  us  it  was 
critical  that  the  MDS  data  be  accurate  because  Vermont  was  making  this 
information  available  to  the  public  as  weU  as  using  it  internally  as  a  normal 
part  of  the  nursing  home  survey  process. 


^^Since  separate  MDS  accuracy  reviews  are  associated  with  states’  Medicaid  programs,  the 
costs  can  be  considered  administrative  expenses.  In  general,  the  federal  government  pays 
75  percent  of  the  cost  for  review  activities  performed  by  skilled  professional  medical 
personnel,  such  as  registered  nurses,  and  50  percent  for  other  personnel  costs.  States  are 
responsible  for  the  remaining  costs. 
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Table  1:  States  with  and  without  MDS  Review  Programs  as  of  January  2001 


State 

Type  of  payment  system 

state 

totals 

States  with  separate  MDS  review  programs 

MDS-based  payment  system 

Indiana,  Iowa,  Maine,  Mississippi, 
Ohio,  Pennsylvania,  South 

Dakota,  Vermont,  Washington, 

West  Virginia 

10 

Planning  to  adopt  MDS-based  payment 
system 

Virginia  (reviews  began 

Aoril  2001) 

1 

States  planning  separate  MDS  review  programs 

MDS-based  payment  system 

Idaho,  Kentucky,  New  Hampshire 

3 

Planning  to  adopt  MDS-based  payment 
system 

Georgia,  Minnesota,®  New  Jersey, 
Utah 

4 

SUBTOTAL 

18 

States  with  no  plans  to  establish  separate  MDS  review  programs 

MDS-based  payment  system 

Colorado, Florida,  Kansas, 
Nebraska,  North  Dakota 

5 

No  MDS-based  payment  system 

Alaska,  Alabama,  Arkansas, 

Arizona,  California,  Connecticut, 
District  of  Columbia,  Delaware, 
Hawaii,  Illinois,®  Louisiana, 
Massachusetts,®  Maryland,^ 
Michigan,  Missouri,  Montana,® 

North  Carolina,  New  Mexico, 
Nevada,  New  York,®  Oklahoma, 
Oregon,  Rhode  Island,  South 
Carolina,  Tennessee,  Texas,® 
Wisconsin,  Wyoming 

28 

Subtotal 

33 

Total 

51 

Note:  States’  decisions  regarding  whether  to  adopt  an  MDS-based  payment  system  and  MDS  review 
program  may  have  changed  since  the  time  of  our  data  collection  (January  2001).  For  example,  a 
Kentucky  official  told  us  that  it  implemented  a  separate  MDS  review  program  in  October  2001,  and 
Montana  has  shifted  to  an  MDS-based  payment  system. 

"Although  these  states  do  not  conduct  a  separate  review  of  MDS  data,  they  do  conduct  separate 
reviews  of  data  that  are  linked  to  their  state’s  Medicaid  payment  system.  For  example,  Texas  has  a 
non-MDS-based  case-mix  payment  system  called  the  Texas  Index  for  Level  of  Effort  that  is  based  on 
a  recipient’s  condition,  ADLs,  and  the  level  of  staff  intervention. 

'’Colorado  and  Maryland  officials  volunteered  that  they  had  conducted  onetime  reviews  of  MDS  data, 
but  are  not  planning  to  regularly  continue  these  reviews.  Colorado’s  state  survey  agency  conducted 
an  MDS  review  of  90  nursing  homes  (40  percent  of  homes)  in  the  summer  of  2000  and  Maryland 
officials  participated  in  a  HCFA-funded  project  to  conduct  on-site  reviews  from  May  through  July  2000 
at  5  percent  of  its  nursing  homes. 

Source:  GAO  survey  of  50  states  and  the  District  of  Columbia. 


To  varying  degrees,  three  m^or  factors  influenced  the  decision  of  33 
states  not  to  establish  separate  MDS  review  programs.  First,  the 
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majority— 28  states— do  not  have  MDS-based  Medicaid  payment  systems. 
Second,  some  states  cited  the  cost  of  conducting  separate  reviews. 

Kansas,  for  example,  reported  a  lack  of  funding  and  staff  resources  as  the 
reason  for  halting  a  brief  period  of  on-site  visits  in  1996  as  a  follow-up  to 
nursing  home  surveys.  Arkansas  as  well  reported  insufficient  staff  for 
conducting  a  separate  review  of  MDS  data.^®  Finally,  officials  in  about  one- 
third  of  the  states  without  separate  MDS  reviews  volunteered  that  they 
had  some  assurance  of  the  accuracy  of  MDS  data  either  because  of 
training  programs  for  persons  responsible  for  completing  MDS 
assessments  or  because  of  the  nursing  home  survey  process.  For 
example,  Missouri  operates  a  state  funded  quality  improvement  project  in 
which  nurses  with  MDS  training  visit  facilities  to  assist  staff  with  the  MDS 
process  and  use  of  quality  indicator  reports.  North  Carolina  also  reported 
that  its  quarterly  training  sessions  provide  MDS  training  to  approximately 
800  providers  a  year.  Regarding  standard  surveys,  Connecticut  and 
Maryland  reported  that  their  nursing  home  survey  teams  reviewed  MDS 
assessments  to  determine  if  they  were  completed  correctly  and  if  the 
assessment  data  matched  surveyor  observations  of  the  resident.  In 
Connecticut,  surveyors  may  also  review  a  sample  of  facility  MDS 
assessments  for  possible  errors  whenever  they  identify  aberrant  or 
questionable  data  on  the  quality  indicator  reports. 

Officials  in  the  10  states  with  separate,  longer  standing  MDS  review 
programs  generally  said  that  the  survey  process  itself  does  not  detect  MDS 


few  of  the  10  states  that  cany  out  separate  MDS  reviews  have  structured  their  programs 
to  reduce  the  costs  of  on-site  reviews.  For  example,  Ohio  uses  off-site  data  analysis  to 
target  a  subset  of  facilities  for  further  on-site  review.  However,  West  Virginia,  which 
conducted  on-site  reviews  until  1998,  cited  a  lack  of  staff  as  the  major  reason  for  switching 
to  an  off-site-only  review  approach. 

^^These  13  states  include:  Connecticut,  Florida,  Kansas,  Maryland,  Michigan,  Missouri, 
Montana,  North  Carolina,  Nevada,  Oregon,  South  Carolina,  Tennessee,  and  Wisconsin. 
Because  states  volunteered  this  information,  there  may  be  other  states  that  conduct  similar 
activities  that  provide  some  assurance  of  the  accuracy  of  MDS  data. 
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accuracy  issues  as  effectively  as  separate  MDS  review  programs.^^  Some 
noted  that  nursing  home  surveyors  do  not  have  time  to  thoroughly  review 
MDS  accuracy  and  often  review  a  smaller  sample  size  than  MDS  reviewers. 
The  surveyors’  primary  focus,  they  indicated,  was  on  quality  of  care  and 
resident  outcomes — not  accuracy  of  MDS  data.  For  example,  surveyors 
would  look  at  whether  the  resident  needed  therapy  and  whether  it  was 
provided.  In  contrast,  the  MDS  reviewer  would  calculate  the  total  number 
of  occupational,  speech,  and  physical  therapy  minutes  to  ensure  that  the 
resident  was  placed  in  the  appropriate  case-mix  category.  Officials  in  Iowa 
similarly  noted  that  surveyors  do  not  usually  cite  MDS  accuracy  as  a 
specific  concern  unless  there  are  egregious  MDS  errors,  again,  because  the 
focus  of  the  survey  process  is  on  quality  of  care. 


States  with  Separate  MDS 
Review  Programs 
Emphasize  On-Site 
Oversight,  but  Also 
Conduct  Off-Site 
Monitoring 


Nine  of  the  10  states  with  separate,  longer  standing  MDS  accuracy  review 
programs  use  on-site  reviews  to  test  the  accuracy  of  MDS  data,  generally 
visiting  all  or  a  significant  portion  of  facilities  in  the  state  at  least  annually, 
if  not  more  frequently.  (See  app.  I  for  a  summary  of  state  on-site  review 
programs.)  Due  to  a  lack  of  staff,  one  state — West  Virginia — limits  its  MDS 
reviews  to  off-site  analysis  of  facility-specific  monthly  data.  Most  of  these 
states  have  been  operating  their  MDS  review  programs  for  7  years  or 
longer  and  developed  them  within  a  year  of  implementing  an  MDS-based 
payment  system.  Three  of  the  nine  states  arrive  at  the  facility 
imannounced  while  the  other  six  provide  advanced  notice  ranging  from  48 
hours  to  2  weeks. 


The  sample  of  facility  MDS  assessments  reviewed  by  each  state  varies 
considerably.  Assessment  sample  sizes  generally  range  from  10  to  40 
percent  of  a  nursing  home’s  total  residents  but  some  states  select  a 
specific  number  of  residents,  not  a  percentage,  and  a  few  specifically 
target  residents  in  particular  case-mix  categories.  For  example,  Indiana 
selects  a  sample  of  40  percent — or  no  less  than  25  residents — ^across  all 


^^Two  of  the  10  states  with  MDS  accuracy  programs  closely  coordinate  their  reviews  with 
state  nursing  home  surveys — Vermont  and  Washington.  In  Vermont,  12  registered  nurses 
separately  conduct  both  the  MDS  accuracy  reviews  and  nursing  home  surveys.  Vermont 
officials  told  us  that  they  had  previously  tried  combining  these  processes  but  decided  to 
separate  them  because  of  the  heavy  workload.  In  Washington,  the  nurses  who  conduct 
nursing  home  surveys  and  MDS  reviews  are  located  in  the  same  department,  and  therefore 
coordinate  closely  by  sharing  reports  and  other  information.  The  quality  assurance  nurses 
who  conduct  the  MDS  reviews  are  surveyor  trained  and  participate  in  nursing  home 
surveys  about  six  times  per  year.  Even  so,  Washington  officials  cited  the  importance  of 
having  a  separate  MDS  review  process  aside  from  the  nursing  home  surveys. 
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major  case-mix  categories,  while  Ohio’s  sample  can  be  based  on  a 
particular  case-mix  category,  such  as  residents  classified  as  “clinically 
complex. Iowa  officials  told  us  that  its  reviewers  select  at  least  25 
percent  of  a  facility’s  residents,  with  a  minimum  of  5  residents,  while 
Pennsylvania  chooses  15  residents  from  each  facility,  regardless  of  case- 
mix  category  or  facility  size.  Some  states  expand  the  resident  sample  when 
differences  between  the  MDS  assessment  and  supporting  documentation 
reach  a  certain  threshold.^^  For  example,  if  the  on-site  review  for  the  initial 
sample  in  Iowa  finds  that  25  percent  or  more  of  the  MDS  assessments  have 
errors,  a  supplemental  random  sample  is  selected  for  review.  While  a  few 
states  limit  their  sample  to  Medicaid  residents  only,  most  select 
assessments  to  review  from  the  entire  nursing  home’s  population. 

On-site  reviews  generally  involve  a  comparison  of  the  documentation  in 
the  resident’s  medical  record  to  the  MDS  assessment  prepared  by  the 
facility.^^  Generally,  the  on-site  process  also  allows  reviewers  to  interview 
nursing  home  staff  and  to  directly  observe  residents,  permitting  a  better 
understanding  of  the  documentation  in  a  resident’s  medical  record  and 
clarifying  any  discrepancies  that  may  exist.  Staff  interviews  and  resident 
observations  can  enhance  the  reviewer’s  understanding  of  the  resident’s 
condition  and  allow  a  more  thorough  MDS  review  than  one  relying 
primarily  on  documentation.  However,  as  the  interval  between  the 
facility’s  MDS  assessment  and  the  on-site  review  increases,  staff 
interviews  and  resident  observations  become  less  reliable  and  more 


^^Generally,  patients  classified  as  clinically  complex  may  have  conditions  such  as  bums, 
pneumonia,  internal  bleeding,  or  dehydration. 

^^States  with  on-site  reviews  generally  define  MDS  errors  as  an  imsupported  MDS 
assessment,  or  they  use  a  stricter  standard  of  an  unsupported  MDS  assessment  that  results 
in  a  change  in  the  resident’s  case-mix  category.  None  of  the  states  identify  whether  an  MDS 
error  results  in  a  quality  indicator  change. 

^^To  strengthen  the  on-site  review  process,  a  few  states — Iowa,  South  Dakota,  and 
Vermont — conduct  interrater  reliability  checks  and  one  of  these  states,  South  Dakota,  also 
conducts  independent  assessments.  During  an  interrater  reliability  check,  two  reviewers 
examine  the  same  MDS  assessment  and  medical  record  separately  and  compare  their 
findings  to  determine  if  they  are  correctly  and  consistently  identi^ung  MDS  errors.  For 
independent  assessments,  reviewers  complete  a  separate  MDS  assessment  using  all  of  the 
available  information  at  the  facility  and  then  compare  it  to  the  original  assessment 
completed  by  the  facility.  In  two  recent  reports,  the  HHS  OIG  also  conducted  independent 
assessments  based  on  medical  record  documentation  for  640  residents.  See  HHS  OIG,  OEI- 
02-99-00040,  Dec.  2000  and  Nursing  Home  Resident  Assessment:  Resource  Utilization 
Groups,  OEI-02-99-0004I  (Washington,  D.C.:  HHS,  Dec.  2000). 
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difficixlt  to  conduct.^^  For  example,  staff  knowledge  of  a  particular  patient 
may  fade  over  time,  the  patient’s  health  status  may  change,  or  the  patient 
may  be  discharged  from  the  facility.  Pennsylvania  officials,  who  reported 
reviewing  assessments  that  were  6  to  12  months  old,  told  us  that  the 
state’s  MDS  reviews  tended  to  identify  whether  the  nursing  home  had 
adequate  documentation.  Reviewing  such  old  assessments  tends  to  focus 
the  review  process  on  the  adequacy  of  the  documentation  rather  than  on 
whether  the  MDS  assessment  was  accurate.^^  Four  of  the  nine  states 
review  assessments  between  30  and  90  days  old,  a  process  that  likely 
increases  the  value  of  interviews  and  observation.  The  combination  of 
interviews  and  observations  can  be  valuable,  but  limiting  reviews  to  only 
recent  MDS  assessments  and  providing  homes  advance  notice  may 
undermine  the  effectiveness  of  on-site  reviews.^®  Under  such 
circumstances,  facilities  have  an  opportunity  to  focus  on  the  accuracy  of 
their  recent  assessments,  particularly  if  the  nursing  home  knows  when 
their  reviews  wiU  occur,  instead  of  adopting  facility-wide  practices  that 
increase  the  accuracy  of  all  MDS  assessments. 

Based  on  their  on-site  reviews,  officials  in  the  nine  states  identified  seven 
areas  as  having  a  high  potential  for  MDS  errors,  with  two  areas  most  often 
identified  as  being  among  the  highest  potential  for  error:  (1)  mood  and 
behavior  and  (2)  nursing  rehabilitation  and  restorative  care.^^  (See  fig.  1.) 
Assessments  of  resident  mood  and  behavior  are  used  to  calculate  quality 
indicators  and,  along  with  nursing  rehabilitation  and  restorative  care,  are 


nine  states  with  on-site  reviews  had  different  criteria  regarding  when  the  assessment 
was  too  old  to  use  interviews  and  observations  as  corroborating  evidence.  For  example, 
one  state  reported  that  interviews  and  observations  become  less  useful  for  an  MDS 
assessment  completed  14  days  prior  to  the  state  review,  while  another  state  cited  180  days. 

^^Similarly,  the  HHS  OIG  acknowledged  that  its  documentation  review  of  MDS  assessments 
up  to  11  months  old  did  not  permit  a  specific  determination  of  why  differences  occurred, 
only  whether  the  MDS  was  consistent  with  the  rest  of  the  medical  record.  See  HHS  OIG, 
OEI-02-99-00041  and  OEI-02-99-00040,  Dec.  2000. 

^®We  have  earlier  reported  that  the  timing  of  some  nursing  home  surveys  makes  them 
predictable,  allowing  facilities  to  mask  certain  deficiencies  if  they  chose  to  do  so.  See 
GAO/HEHS-00-197,  p.  11. 

^^Nursing  rehabilitation  and  restorative  care  are  interventions  that  assist  or  promote  the 
resident’s  ability  to  attain  his  or  her  maximum  functional  potential.  Some  examples  include 
passive  or  active  range  of  motion  movements,  amputation  care,  and  splint  or  brace 
assistance. 
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often  important  in  determining  nursing  home  payments.^®  CMS  indicated 
that  several  of  the  MDS  elements  cited  in  figure  1  were  also  identified  by  a 
CMS  contractor  as  areas  of  concern.  Officials  in  most  states  with  separate 
on-site  review  programs  told  us  that  errors  discovered  during  their  on-site 
reviews  often  resulted  from  differences  in  clinical  interpretation  or 
mistakes,  such  as  a  misunderstanding  of  MDS  definitions  by  those 
responsible  for  completing  MDS  assessments.  Officials  in  only  four  of  the 
nine  states  were  able  to  tell  us  whether  the  errors  identified  in  their  MDS 
reviews  on  average  resulted  in  a  case-mix  category  that  was  too  high  or 
too  low.  Two  of  these  states  reported  roughly  equal  numbers  of  MDS 
errors  that  inappropriately  placed  a  resident  in  either  a  higher  or  lower 
case-mix  category;  a  third  indicated  that  errors  more  often  resulted  in 
higher  payments;  and  a  fourth  found  that  errors  typically  resulted  in 
payments  that  were  too  low.  None  of  the  nine  states  track  whether  quality 
indicator  data  were  affected  by  MDS  errors. 


^^For  example,  2  of  the  24  quality  indicators  are  based  on  behavior  areas  assessed  in  the 
MDS,  such  as  residents  being  verbally  abusive,  physically  abusive,  or  showing  symptoms  of 
depression. 
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Figure  1:  MDS  Elements  Identified  By  Nine  States  As  Having  High  Potential  for  MDS 
Errors 


Number  of  states 


Identified  as  1  of  top  3  high  potential  areas 
Identified  as  high  potential  area 


Note:  We  asked  states  to  identify  areas  of  the  MDS  assessment  that  have  a  high  potential  for  MDS 
errors.  State  responses  were  included  in  this  figure  if  two  or  more  states  identified  an  area  as  “high 
potential.” 

“Staff  record  the  number  of  days  and  total  minutes  of  therapy,  such  as  physical  or  occupational 
therapy,  received  by  a  resident  in  the  last  7  days. 

‘’Staff  record  the  number  of  days  during  the  last  14-day  period  in  which  a  physician  has  examined  the 
resident  or  changed  the  care  directions  for  the  resident.  The  latter  is  known  as  physician  orders. 

‘'Staff  members  record  scheduled  times  each  day  that  they  perform  any  of  the  following  tasks;  (1)  take 
the  resident  to  the  bathroom,  (2)  give  the  resident  a  urinal,  or  (3)  remind  the  resident  to  go  to  the 
bathroom. 

Source:  Interviews  with  officials  from  nine  states  with  separate  on-site  review  programs  in  operation 
as  of  January  2001 . 
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Two  of  the  10  states  with  MDS  review  programs  were  able  to  tell  us  the 
amount  of  Medicaid  recoupments  resulting  from  inaccurate  MDS 
assessments.  From  state  fiscal  years  1994  through  1997,  South  Dakota 
officials  reported  that  the  state  had  recouped  about  $360,000  as  a  result  of 
recalculating  nursing  home  payments  after  MDS  reviews.  West  Virgima 
received  $1  million  in  1999  related  to  MDS  errors  for  physical  therapy 
discovered  during  a  1995  on-site  review  at  a  nursing  home.  Officials  in  five 
additional  states  told  us  that  they  recalculate  nursing  home  payments 
when  MDS  errors  are  found,  but  could  not  provide  the  amount  recovered.^^ 

Of  the  10  states  with  longer  standing  MDS  review  programs,  four  use  off¬ 
site  analyses  to  supplement  their  on-site  reviews,  while  one  state  relies  on 
off-site  analyses  exclusively.  Both  Maine  and  Washington  examine  MDS 
data  off-site  to  monitor  changes  by  facility  in  the  mix  of  residents  across 
case-mix  categories.  Such  changes  may  help  identify  aberrant  or 
inconsistent  patterns  that  may  indicate  the  need  for  further  investigation. 
Ohio,  a  state  with  approximately  1,000  facilities — more  than  any  other 
state  that  conducts  MDS  reviews — analyzes  data  off-site  to  identify 
facilities  with  increased  Medicaid  payments  and  changes  in  case-mix 
categories  to  select  the  approximately  20  percent  of  facilities  visited  each 
year.^°  West  Virginia  has  eliminated  its  on-site  reviews  and  now  focuses 
solely  on  analyzing  monthly  reports  for  its  141  facilities — for  example, 
significant  changes  in  case-mix  categories  or  ADLs  across  consecutive 
MDS  assessments.  In  addition  to  informally  sharing  results  of  off-site 
reviews  with  the  state  nursing  home  surveyors,  West  Virginia  is  trying  to 
formalize  a  process  in  which  off-site  reviews  could  trigger  additional  on¬ 
site  or  off-site  documentation  reviews. 


the  time  of  our  interviews,  three  states  did  not  recalculate  Medicaid  payments  as  a 
result  of  errors  found  during  MDS  reviews— Maine,  Pennsylvania,  and  Iowa. 

^^Although  Virginia  had  not  begun  its  reviews  at  the  time  of  our  data  collection,  state 
officials  told  us  that  they  planned  to  use  off-site  data  analysis  to  target  approximately  20 
facilities — 7  percent — ^per  month  for  on-site  review. 
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States  Attempt  to 
Improve  MDS  Data 
Accuracy  through  On- 
Site  Reviews, 

Training,  and  Other 
Remedies 


Officials  in  the  nine  states  with  on-site  review  programs  consistently  cited 
three  features  of  their  review  programs  that  strengthened  the  ability  of 
nursing  home  staff  to  complete  accurate  MDS  assessments  and  thus 
decrease  errors:  (1)  the  actual  presence  of  reviewers,  (2)  provider 
education,  and  (3)  remedies  that  include  corrective  action  plans  and 
financial  penalties.  On-site  reviews,  for  example,  underscore  the  state’s 
interest  in  MDS  accuracy  and  provide  an  opportunity  to  train  and  coach 
those  who  are  responsible  for  completing  MDS  assessments.  Similarly,  the 
errors  discovered  during  on-site  reviews  guide  the  development  of  more 
formal  training  sessions  that  are  offered  by  the  state  outside  of  the  nursing 
home.  Requiring  nursing  homes  to  prepare  corrective  action  plans  and 
imposing  financial  penalties  signal  the  importance  of  MDS  accuracy  to 
facilities  and  are  tools  to  improve  the  accuracy  of  the  MDS  data.  As  a 
result  of  these  efforts,  some  states  have  been  able  to  show  a  notable 
decrease  in  their  overall  error  rates. 


Most  of  the  nine  states  view  on-site  visits  and  training  as  interrelated 
elements  that  form  the  foxmdation  of  their  MDS  review  programs.  State 
officials  said  that  nursing  homes  pay  more  attention  to  properly 
documenting  and  completing  the  MDS  assessments  because  reviewers 
visit  the  facilities  regularly.  On-site  visits  also  allow  reviewers  to  discuss 
MDS  documentation  issues  or  requirements  with  staff,  providing  an 
opportunity  for  informal  MDS  training.  For  example,  Indiana  officials  told 
us  that  2  to  3  hours  of  education  are  a  routine  part  of  each  facility’s  MDS 
review.  Noting  the  high  staff  turnover  rates  in  nursing  homes,  many  states 
reported  that  frequent  training  for  the  staff  responsible  for  completing 
MDS  assessments  is  critical.^^  Officials  in  seven  of  the  nine  states  with  on¬ 
site  reviews  told  us  that  high  staff  turnover  was  one  of  the  top  three 
factors  contributing  to  MDS  errors  in  their  states.  In  addition,  many  of  the 


recently  testified  on  the  problem  of  nurse  and  nurse  aide  retention  in  a  range  of  health 
care  settings,  including  nursing  homes.  See  Nursing  Workforce:  Recruitment  and 
Retention  of  Nurses  and  Nurse  Aides  Is  a  Growing  Concern  (GAO-01-750T,  May  17, 2001). 
In  addition,  the  HHS  OIG  recently  reported  that  about  60  percent  of  MDS  coordinators  had 
worked  1  year  or  less  in  that  role  at  their  current  nursing  home  and  over  65  percent  had  no 
prior  experience  as  an  MDS  coordinator.  See  HHS  OIG,  OEI-02-99-00040,  Dec.  2000. 
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reasons  cited  for  MDS  errors — such  as  a  misunderstanding  of  MDS 
definitions  and  other  mistakes — ^reinforce  the  need  for  training.^^ 

States  with  on-site  reviews  use  the  process  to  guide  provider  education 
activities — ^both  on-site  and  off-site.  For  example,  during  Pennsylvania’s 
annual  MDS  reviews  of  all  nursing  homes,  state  reviewers  determine  the 
types  of  training  needed.  According  to  state  officials,  the  state  uses  the 
results  of  these  reviews  to  shape  and  provide  facility-specific  training,  if  it 
is  needed,  within  a  month  of  the  review  and  subsequently  conducts  a 
foUow-up  visit  to  see  if  the  facility  is  improving  in  these  areas.  They 
indicated  that  all  685  homes  visited  during  2000,  the  first  year  of  this 
approach,  were  provided  with  some  type  of  training.  To  improve  MDS 
accuracy,  several  states  also  provide  voluntary  training  opportunities 
outside  of  the  nursing  home.  Maine,  Iowa,  Indiana,  and  South  Dakota,  for 
example,  provide  MDS  training  regularly  throughout  the  state,  rotating  the 
location  of  the  training  by  region  so  that  it  is  accessible  to  staff  from  all 
facilities. 

While  states  generally  emphasized  on-site  reviews  and  training  as  the 
primary  ways  to  improve  the  accuracy  of  the  MDS  data,  some  reported 
that  they  have  also  instituted  certain  remedies,  such  as  corrective  action 
plans  and  financial  penalties.  Indiana  and  Pennsylvania,  for  example, 
require  facilities  to  submit  a  corrective  action  plan  detailing  how  the 
facility  will  address  errors  identified  during  an  on-site  review.  Two 
states — ^Maine  and  Indiana — ^impose  financial  penalties.^  Maine  has 
instituted  financial  penalties  for  recurring  serious  errors,  collecting 
approximately  $390,000  since  late  1995.  Maine  also  requires  facilities  with 
any  MDS  errors  that  result  in  a  case-mix  category  change  to  complete  and 


^^HCFA  provided  guidance  in  March  and  July  2001  to  facilities  regarding  the  completion  of 
MDS  assessments.  HCFA  last  published  similar  guidance  in  August  1996.  A  few  state 
officials  noted  the  long  lapse  in  the  publication  between  the  two  guides  and  told  us  that 
clearer  and  more  timely  guidance  on  MDS  definitions  was  needed.  However,  CMS’  Long 
Term  Care  Facility  Resident  Assessment  Instrument  User's  Manual^  which  provides 
guidance  on  completing  MDS  assessments,  has  not  been  updated  since  1995. 

Vermont  and  Washington  also  told  us  that  financial  penalties  are  an  available  remedy,  but 
had  not  imposed  them  as  of  early  2001. 
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submit  a  corrected  MDS  assessment  for  the  resident.®^  While  Indiana 
imposes  financial  penalties,  it  does  not  view  them  as  the  primary  tool  for 
improving  MDS  accuracy.^^  Rather,  officials  attributed  a  decrease  in  MDS 
errors  to  the  education  of  providers  and  the  on-site  presence  of  reviewers. 
Other  remedies  cited  by  states  include  conducting  more  frequent  on-site 
MDS  reviews  and  referring  suspected  cases  of  fraud  to  their  state’s 
Medicaid  Fraud  Control  Unit. 

Five  of  the  nine  states  that  conduct  on-site  MDS  reviews  told  us  that  their 
efforts  have  resulted  in  a  notable  decrease  in  MDS  errors  across  all 
facilities  since  the  implementation  of  their  review  programs.  (See  table  2.) 
South  Dakota  officials,  for  example,  reported  that  the  percentage  of 
assessments  with  MDS  errors  across  facilities  had  decreased  from 
approximately  85  percent  to  10  percent  since  the  implementation  of  the 
state’s  MDS  review  program  in  1993.  Similarly,  Indiana  reported  a  decrease 
in  the  statewide  average  error  rate  from  75  percent  to  30  percent  of 
assessments  in  1  year’s  time.  Four  states  could  not  provide  these  data.  In 
calculating  these  decreases,  three  of  the  five  states — ^Indiana,  Maine,  and 
South  Dakota — define  MDS  errors  as  an  unsupported  MDS  assessment 
that  caused  the  case-mix  category  to  be  inaccurate.^®  Iowa’s  definition, 
however,  includes  MDS  elements  that  are  not  supported  by  medical  record 
documentation,  observation,  or  interviews,  regardless  of  whether  the  MDS 
error  changed  the  case-mix  category.  Similarly,  while  Pennsylvania  does 
not  limit  errors  to  those  that  changed  the  case-mix  category,  the  state 


Maine,  facilities  are  instructed  to  follow  CMS’  correction  policy  guidelines  for  MDS 
errors  that  do  not  result  in  a  case-mix  category  change.  In  commenting  on  a  draft  of  this 
report,  CMS  noted  the  development  and  implementation  of  its  policy,  which  provided  a 
new  mechanism  for  facilities  to  correct  inaccurate  information  in  the  MDS  database.  This 
new  policy  has  significantly  decreased  the  ability  of  facilities  to  submit  certain  types  of 
inaccurate  MDS  data,  such  as  entering  a  “5”  for  a  particular  MDS  element,  when  tiie  only 
available  choices  are  “1-4.”  Under  this  policy,  CMS  has  seen  a  reduction  of  approximately 
66  percent  in  the  proportion  of  records  in  the  database  containing  invalid  data  values. 

®®Indiana  imposes  financial  penalties  if  more  than  35  percent  of  a  facility’s  MDS 
assessments  have  errors.  State  officials  told  us  that  very  few  facilities — roughly  3  to  4  each 
quarter—have  errors  that  are  significant  enough  to  trigger  financial  penalties. 

^®In  Maine,  only  a  subset  of  these  case-mix  category  changes  is  used  to  calculate  an  error 
rate. 
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defines  errors  as  a  subset  of  MDS  elements  that  are  not  supported  by  the 
medical  record.^^ 


Table  2:  MDS  Assessments  with  Errors  in  Five  States  with  On-Site  MDS  Review 
Programs  (In  percent) 

State 

Initial  MDS 
error  rate 

Subsequent 
MDS  error 
rate 

Time  of  initial  and  subsequent 
error  rate 

Indiana 

75 

30 

1999,  2000 

Iowa 

32 

22 

July,  December  2000 

Maine® 

21 

10 

1995, 2000 

Pennsylvania 

20 

15 

2000,  2001 

South  Dakota 

85 

10 

1993, 1998 

‘'Errors  that  result  in  changes  for  a  subset  of  case-mix  categories  were  used  to  calculate  these  error 
rates. 

Source:  Data  provided  by  Indiana,  Iowa,  Maine,  Pennsylvania,  and  South  Dakota. 


CMS’  MDS  Review 
Program  Could  Better 
Leverage  Existing 
State  and  Federal 
Accuracy  Activities 


Following  implementation  of  Medicare’s  MDS-based  payment  system  in 
1998,  HCFA  began  building  the  foundation  for  its  own  separate  review 
program — distinct  from  state  efforts — ^to  help  ensure  the  accuracy  of  MDS 
data.  In  the  course  of  developing  and  testing  accuracy  review  approaches, 
its  contractor  found  widespread  MDS  errors  that  resulted  in  a  change  in 
Medicare  payment  categories  for  67  percent  of  the  resident  assessments 
sampled.  In  September  2001,  CMS  awarded  a  new  contract  to  implement  a 
nationwide  MDS  review  program  over  a  2-  to  3-year  period.^®  Despite  the 
benefits  of  on-site  reviews,  as  demonstrated  by  states  with  separate  review 
programs,  the  current  plan  involves  conducting  on-site  reviews  in  fewer 
than  200  of  the  nation’s  17,000  nursing  homes  each  year.  In  addition,  the 
contractor’s  combined  on-site  and  off-site  reviews  to  evaluate  MDS 
accuracy  will  involve  only  about  1  percent  of  the  approximately  14.7 
million  MDS  assessments  expected  to  be  prepared  in  2001.  In  contrast, 
states  that  conduct  separate  on-site  MDS  reviews  typically  visit  all  or  a 


^^Pennsylvania  reviews  only  those  MDS  elements  that  have  a  positive  response.  For 
example,  if  a  facility  responded  “no”  or  left  an  MDS  element  blank,  that  item  would  not  be 
reviewed  for  accuracy,  even  if  it  could  affect  the  case-mix  category  for  that  particular 
resident. 

^®CMS  refers  to  the  contractor  responsible  for  this  program  as  the  data  assessment  and 
verification  (DAVE)  contractor. 
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significant  portion  of  their  nursing  homes  and  generally  examine  from  10 
to  40  percent  of  assessments.  While  CMS’  approach  may  yield  some  broad 
sense  of  the  accuracy  of  MDS  assessments  on  an  aggregate  level,  it  may  be 
insufficient  to  help  ensure  the  accuracy  of  MDS  assessments  in  most  of  the 
nation’s  nursing  homes.  At  present,  it  does  not  appear  that  CMS  plans  to 
leverage  the  considerable  resources  already  devoted  to  state  nursing  home 
surveys  and  states’  separate  MDS  review  programs  that  together  entail  a 
routine  on-site  presence  in  all  nursing  homes  nationwide.  Nor  does  it  plan 
to  more  systematically  evaluate  the  performance  of  state  survey  agencies 
regarding  MDS  accuracy  through  its  own  federal  comparative  surveys. 
Finally,  CMS  is  not  requiring  nursing  homes  to  provide  documentation  for 
the  full  MDS  assessment,  which  could  undermine  the  efficacy  of  its  MDS 
reviews. 


Testing  of  MDS  Accuracy 
Approaches  Identified 
Widespread  Accuracy 
Problems 


In  September  1998,  HCFA  contracted  with  Abt  Associates  to  develop  and 
test  various  on-site  and  off-site  approaches  for  verifying  and  improving  the 
accuracy  of  MDS  data.  Two  of  the  approaches  resembled  state  on-site 
MDS  reviews  and  the  off-site  documentation  reviews  performed  by  CMS 
contractors  that  review  Medicare  claims.^®  Another  approach  used  off-site 
data  analysis  to  target  facilities  for  on-site  review.'*®  To  determine  the 
effectiveness  of  the  approaches  tested  in  identifying  MDS  inaccuracies, 
Abt  compared  the  errors  found  under  each  approach  to  those  found  in  its 
“reference  standard” — ^independent  assessments  performed  by  MDS- 
trained  nurses  hired  by  Abt  for  approximately  600  residents  in  30  facilities 


^^Similar  to  the  separate  MDS  reviews  conducted  by  the  states,  Abt  reviewed  a  subset  of 
MDS  items  at  a  sample  of  nursing  homes  that  met  certain  criteria,  e.g,,  they  were  important 
in  determining  case-mix  categories  or  calculating  quality  indicators  or  were  suspected  of 
being  underreported.  Abt  reviewers  used  information  from  medical  records  as  well  as 
interviews  and  observations  with  staff  and  residents  to  determine  whether  the  selected 
items  on  the  MDS  assessments  were  accurate. 

“^^One  off-site  approach  tested  relied  on  analyzing  certain  MDS  “trigger”  items,  such  as 
pneumonia,  that  are  likely  to  be  in  error  when  found  in  a  certain  pattern  on  two 
consecutive  MDS  assessments  for  the  same  resident.  Off-site  data  analysis  under  this 
approach  could  be  used  to  identify  facilities  for  on-site  review  that  have  a  high  proportion 
of  residents  shown  as  having  pneumonia — one  potential  trigger  item — across  two  or  more 
MDS  assessments. 
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in  three  states/^  Abt  found  errors  in  every  facility,  with  little  variation  in 
the  percentage  of  assessments  with  errors  across  facilities.  On  average, 
the  errors  found  affected  case-mix  categories  in  67  percent  of  the  sampled 
Medicare  assessments.  Abt  concluded  that  the  errors  did  not  result  in 
systematic  overpayments  or  imderpayments  to  facilities  even  though  there 
were  more  errors  that  placed  residents  in  too  high  as  opposed  to  too  low  a 
case-mix  category.  Abt  did  not  determine,  however,  the  extent  to  which 
errors  affected  quality  indicators. 

Due  to  the  prevalence  of  errors,  Abt  recommended  a  review  program  that 
included  periodically  visiting  all  facilities  during  the  program’s  first  several 
years.  Recognizing  the  expense  of  visiting  every  facility,  however,  Abt  also 
recommended  eventually  transitioning  to  the  use  of  off-site  mechanisms  to 
target  facilities  and  specific  assessments  for  on-site  review.  Abt  also  made 
recommendations  to  address  the  underlying  causes  of  MDS  errors: 
simplifying  the  MDS  assessment  tool,  clarifying  certain  MDS  definitions 
(particularly  for  ADLs),  and  improving  MDS  training  for  facilities.'*^ 


The  Federal  MDS  Review 
Program  Is  Too  Limited  to 
Evaluate  State-Level 
Accuracy  Assurance 
Efforts 


Building  on  the  work  of  Abt  Associates,  in  the  summer  of  2000,  the  agency 
began  formulating  its  own  distinct  nationwide  review  program  to  address 
long-term  MDS  monitoring  needs.  The  agency  developed  a  request  for 
proposal  for  MDS  data  assessment  and  verification  activities  and  sought 
proposals  fi"om  its  12  program  safeguard  contractors.'*^  On  September  28, 
2001,  CMS  awarded  a  3-year  contract  for  approximately  $26  million  to 


'**The  nurses  conducted  assessments  over  several  days  and  shifts  using  all  available 
documentation — medical  record  reviews,  interviews,  and  observations — ^to  replicate  as 
closely  as  possible  the  observation  period  the  facility  used  to  make  its  assessments  of 
those  same  residents.  Because  Abt  found  too  few  assessments  meeting  its  original 
criteria — completed  by  the  facility  up  to  14  days  prior  to  the  visits — it  augmented  its 
sample  with  assessments  that  were  up  to  35  days  old, 

'*^Similar  to  Abt,  the  HHS  OIG  concluded  that  differences  found  between  MDS  assessments 
and  the  supporting  documentation  indicated  confusion  or  difficulties  with  the  MDS 
assessment  instrument  and  the  need  for  enhanced  training.  The  HHS  OIG  found  differences 
in  76  percent  of  the  Medicare  assessments  reviewed.  ADLs  and  the  number  of  minutes 
recorded  for  therapy,  specifically  occupational  and  physical  therapy,  provided  the  greatest 
source  of  differences. 

'*^Program  safeguard  contractors  were  authorized  by  the  Health  Insurance  Portability  and 
Accountability  Act  of  1996,  which  allowed  HCFA  to  contract  with  specialized  entities  to 
identify  program  integrity  concerns.  See  42  U.S.C.  §  1395ddd.  In  May  1999,  HCFA  selected  a 
pool  of  12  contractors  that  can  bid  on  proposed  contracts  covering  these  types  of  activities. 
See  Medicare:  Opportunities  and  Giollenges  in  Contracting  for  Program  Safeguards 
(GAO-01-616,  May  18, 2001). 
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Computer  Sciences  Corporation.  The  contract  calls  for  the  initiation  of  on¬ 
site  and  off-site  reviews  by  late  spring  2002,  but  the  full  scope  of  MDS 
review  activities  will  not  be  underway  until  the  second  year  of  the 
contract.  ^  (See  table  3.) 


Table  3:  Implementation  Schedule  for  CMS’  MDS  Accuracy  Review  Program 

Phase 

Time  period 

Review  activities 

Developmental 

October  2001 
through  May  2002 

•  Test  a  combination  of  the  most  promising 
components  from  Abt’s  earlier  assessment 
of  various  on-site  and  off-site  approaches. 

•  Recommend  the  appropriate  balance 
between  on-site  and  off-site  reviews. 

•  Identify  and  develop  new  approaches  for 
monitoring  MDS  accuracy. 

•  Begin  to  identify  communication  and 
collaboration  strategies  for  federal  and 
state  accuracy  reviews,  such  as 
coordinating  with  states. 

Initial 

implementation 

April  2002  through 
September  2002 

•  Begin  conducting  on-site  and  off-site 
accuracy  reviews. 

•  Continue  to  evaluate  the  efficacy  of  the 
accuracy  review  approaches  being 
implemented  and  identify  areas  of  risk. 

•  Conduct  ongoing  data  surveillance,  such  as 
monitoring  and  identifying  trends  in 
payments  based  on  MDS  data.^ 

Full  Implementation 

October  2002 
through 

September  2003 

•  Perform  ongoing  data  analysis  and  the  full 
scope  of  data  assessment  and  verification 
activities.^ 

•  Implement  training  and  education  activities 
to  ensure  that  those  responsible  for  MDS 
data  understand  and  accurately  complete 
MDS  assessments.  This  approach  is 
expected  to  include  a  method  for 
communicating  how  the  contractor  will 
continually  refine  and  improve  accuracy 
review  processes. 

Note:  The  contract  covers  1  year  with  two  additional  1-year  options.  Currently,  full  implementation 
would  occur  in  the  second  year  of  the  contract.  The  third  year  of  the  contract  may  also  include  on-site 
enforcement  surveys  and  special  studies  concerning  the  accuracy  of  reported  Medicare  and  Medicaid 
data. 

^Although  the  contractor  will  first  focus  on  conducting  MDS  accuracy  activities,  the 
contractor  is  also  required  to  establish  a  review  program  for  the  Outcome  and  Assessment 
Information  Set  (OASIS),  the  data  used  as  the  basis  for  home  health  payments  and  quality 
measures. 
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Tor  example,  one  of  the  contractor’s  tasks  is  to  analyze  MDS  data  reported  by  nursing  homes  that 
serve  Medicare  beneficiaries  to  determine  whether  differences  in  case-mix  categories  relate  to 
changes  in  the  patient’s  health  status  or  changes  in  how  providers  are  reporting  MDS  data. 

Tor  example,  while  continuing  on-site  and  off-site  MDS  reviews,  the  contractor  will  also  be  required 
to  calculate  error  rates  for  paid  claims  for  Medicare-covered  services. 

Source:  DAVE  contract  statement  of  work  for  CMS’  review  program  for  MDS  accuracy. 

Despite  this  broad  approach,  the  contractor  is  not  specifically  tasked  with 
assessing  the  adequacy  of  each  state’s  MDS  reviews.  Instead,  it  is  required 
to  develop  a  strategy  for  coordinating  its  review  activities  with  other  state 
and  federal  oversight,  such  as  the  selection  of  facilities  and  the  timing  of 
visits,  to  avoid  unnecessary  overlap  with  routine  nursing  home  surveys  or 
states’  separate  MDS  review  programs.  This  approach  does  not  appear  to 
build  on  the  benefits  of  on-site  visits  that  are  dready  occurring  as  part  of 
state  review  activities.  Rather,  the  contract  specifies  independent  federal 
on-site  and  off-site  reviews  of  roughly  1  percent  of  the  approximately 
14.7  million  MDS  assessments  expected  to  be  prepared  in  2001 — 80,000 
during  the  first  contract  year  and  130,000  per  year  thereafter.'^^  The 
contractor,  however,  tentatively  recommended  that  the  majority  of 
reviews,  about  90  percent,  be  conducted  off-site.  According  to  CMS,  these 
off-site  reviews  could  include  a  range  of  activities,  such  as  the  off-site 
targeting  approaches  developed  by  Abt  or  medical  record  reviews  similar 
to  those  conducted  by  CMS  contractors  for  purposes  of  reviewing 
Medicare  claims.  In  addition,  the  contractor  is  expected  to  conduct  a 
range  of  off-site  data  analyses  that  could  include  a  large  number  of  MDS 
assessments.  The  remaining  10  percent  of  MDS  assessments — 
representing  fewer  than  200  of  the  nation’s  17,000  nursing  homes — would 
be  reviewed  on-site  each  year.  This  limited  on-site  presence  is  inconsistent 
with  Abt’s  earlier  recommendation  regarding  the  benefits  of  on-site 
reviews  in  detecting  accuracy  problems,  and  with  the  view  of  almost  all  of 
the  states  with  separate  MDS  review  programs  that  an  on-site  presence  at 
a  significant  number  of  their  nursing  homes  is  central  to  their  review 
efforts. 

While  CMS’  approach  may  yield  some  broad  sense  of  the  accuracy  of  MDS 
assessments  on  an  aggregate  level,  it  appears  to  be  insufficient  to  provide 
confidence  about  the  accuracy  of  MDS  assessments  in  the  vast  bulk  of 
nursing  homes  nationwide.  Given  the  substantial  resources  invested  in  on¬ 
site  nursing  home  visits  associated  with  standard  surveys  or  states’ 


'^^The  reviews  would  encompass  assessments  from  all  payer  sources.  According  to  CMS, 
the  number  of  assessments  to  be  reviewed  is  a  target  that  is  subject  to  change. 
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separate  MDS  review  programs,  CMS’  MDS  review  program  could  view 
states’  routine  presence  as  the  cornerstone  of  its  program  and  instead 
focus  its  efforts  on  ensuring  the  adequacy  of  state  reviews.  CMS  could 
build  on  its  established  federal  monitoring  survey  process  for  nursing 
home  oversight.  The  agency  is  required  by  statute  to  annually  resurvey  at 
least  5  percent  of  all  nursing  homes  that  participate  in  Medicare  and 
Medicaid.  One  of  the  ways  CMS  accomplishes  this  requirement  is  by 
conducting  nursing  home  comparative  surveys  to  independently  assess  the 
states’  performance  in  their  nursing  home  survey  process.  During  a 
comparative  survey,  a  federal  team  independently  surveys  a  nursing  home 
recently  inspected  by  a  state  in  order  to  compare  and  contrast  the  results. 
These  federal  comparative  surveys  have  been  found  to  be  most  effective 
when  completed  in  close  proximity  to  the  state  survey  and  involve  the 
same  sample  of  nursing  home  residents  to  the  maximum  extent  possible. 
Abt  also  attempted  to  review  recently  completed  MDS  assessments. 

Finally,  a  potential  issue  that  could  undermine  the  efficacy  of  the  federal 
MDS  accuracy  reviews  involves  the  level  of  documentation  required  to 
support  an  MDS  assessment.  CMS  requires  specific  documentation  for 
some  MDS  elements,  but  officials  said  that  the  MDS  itself — which  can 
simply  consist  of  checking  off  boxes  or  selecting  multiple  choice  answers 
on  the  assessment  foim — generally  constitutes  support  for  the  assessment 
without  any  additional  documentation.  CMS  officials  consider  the  MDS 
assessment  form  to  have  equal  weight  with  the  other  components  of  the 
medical  record,  such  as  physician  notes  and  documentation  of  services 
provided.  As  a  result,  CMS  asserts  that  the  assessment  must  be  consistent 
with,  but  need  not  duplicate,  the  medical  record.  In  contrast,  most  of  the 
nine  states  with  separate  on-site  review  programs  require  that  support  for 
each  MDS  element  that  they  review  be  independently  documented  in  the 
medical  record.  State  officials  told  us  that  certain  MDS  elements,  such  as 
ADLs,  are  important  to  thoroughly  document  because  they  require 
observation  of  many  activities  by  different  nursing  home  staff  over  several 
days.  As  a  result,  some  of  these  states  require  the  use  of  separate  flow 
charts  or  tables  to  better  document  ADLs.  Similarly,  some  states  require 
documentation  for  short-term  memory  loss  rather  than  accepting  a  nursing 
home’s  assertion  that  a  resident  has  this  condition.  CMS’  training  manual 
describes  several  appropriate  tests  for  identifying  memory  loss,  such  as 
having  a  resident  describe  a  recent  event.  In  one  of  its  December  2000 
reports,  the  HHS  OIG  recommended  that  nursing  homes  be  required  to 
establish  an  “audit  trail”  to  support  certain  MDS  elements.  HCFA 
disagreed,  noting  that  it  does  not  expect  all  information  in  the  MDS  to  be 
duplicated  elsewhere  in  the  medical  record.  However,  given  the  uses  of 
MDS  data,  especially  in  ac^usting  nursing  home  payments  and  producing 
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quality  indicators,  documenting  the  basis  for  the  MDS  assessments  in  the 
medical  record  is  critical  to  assessing  their  accuracy. 


Conclusions 


In  complying  with  federal  nursing  home  participation  and  quality 
requirements,  about  17,000  nursing  homes  were  expected  to  produce 
almost  15  million  MDS  assessments  during  2001  on  behalf  of  their 
residents.  This  substantial  investment  of  nursing  home  staff  time 
contributes  to  multiple  functions,  including  establishing  patient  care  plans, 
assisting  with  quality  oversight,  and  setting  nursing  home  payments  that 
accoimt  for  variation  in  resident  care  needs.  While  some  states, 
particularly  those  with  MDS-based  Medicaid  payment  systems,  stated  that 
ensuring  MDS  accuracy  requires  establishing  a  separate  MDS  review 
program,  many  others  rely  on  standard  nursing  home  surveys  to  assess  the 
data’s  accuracy.  Flexibility  in  designing  accuracy  review  programs  that  fit 
specific  state  needs,  however,  should  not  preclude  achieving  the  important 
goal  of  ensuring  accountability  across  state  programs.  It  is  CMS’ 
responsibility  to  consistently  ensure  that  states  are  fulfilling  statutory 
requirements  to  accurately  assess  and  provide  for  the  care  needs  of 
nursing  home  residents. 

The  level  of  federal  financial  support  for  state  MDS  accuracy  activities  is 
already  substantial.  The  federal  government  pays  up  to  75  percent  of  the 
cost  of  separate  state  MDS  review  activities  and  in  fiscal  year  2001 
contributed  $278  million  toward  the  cost  of  the  state  nursing  home  survey 
process,  which  is  intended  in  part  to  review  MDS  accuracy.  Instead  of 
establishing  a  distinct  but  limited  federal  review  program,  reorienting  the 
thrust  of  its  review  program  in  order  to  complement  ongoing  state  MDS 
accuracy  efforts  could  prove  to  be  a  more  efficient  and  effective  means  to 
achieve  CMS’  stated  goals.  Such  a  shift  in  focus  should  include  (1)  taking 
full  advantage  of  the  periodic  on-site  visits  already  conducted  at  eveiy 
nursing  home  nationwide  through  the  routine  state  survey  process, 

(2)  ensuring  that  the  federal  MDS  review  process  is  designed  and 
sufficient  to  consistently  assess  the  performance  of  all  states’  reviews  for 
MDS  accuracy,  and  (3)  providing  additional  guidance,  training,  and  other 
technical  guidance  to  states  as  needed  to  facilitate  their  efforts.  With  its 
established  federal  monitoring  system  for  nursing  home  surveys — 
especially  the  comparative  survey  process — ^that  helps  assess  state 
performance  in  conducting  the  nursing  home  survey  process,  CMS  has  a 
ready  mechanism  in  place  that  it  can  use  to  systematically  assess  state 
performance  for  this  important  task.  Finally,  to  help  improve  the 
effectiveness  of  MDS  review  activities,  CMS  should  take  steps  to  ensure 
that  each  MDS  assessment  is  adequately  supported  in  the  medical  record. 
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Recommendations  for 
Executive  Action 


Agency  and  State 
Comments  and  Our 
Evaluation 


With  the  goal  of  complementing  and  leveraging  the  considerable  federal 
and  state  resources  already  devoted  to  nursing  home  surveys  and  to 
separate  MDS  accuracy  review  programs,  we  recommend  that  the 
administrator  of  CMS 

review  the  adequacy  of  current  state  efforts  to  ensure  the  accuracy  of  MDS 
data,  and  provide,  where  necessary,  additional  guidance,  training,  and 
technical  assistance; 

monitor  the  adequacy  of  state  MDS  accuracy  activities  on  an  ongoing 
basis,  such  as  through  the  use  of  the  established  federal  comparative 
survey  process;  and 

provide  guidance  to  state  agencies  and  nursing  homes  that  sufficient 
evidentiary  documentation  to  support  the  full  MDS  assessment  be 
included  in  residents’  medical  records. 


We  provided  a  draft  of  this  report  to  CMS  and  the  10  states  with  separate 
MDS  accuracy  programs  for  their  review  and  comment.  (See  app.  II  for 
CMS’  comments.)  CMS  agreed  with  the  importance  of  assessing  and 
monitoring  the  adequacy  of  state  MDS  accuracy  efforts.  CMS  also 
recognized  that  the  MDS  affects  reimbursement  and  care  planning  and  that 
it  is  essential  that  the  assessment  data  reflect  the  resident’s  health  status 
so  that  the  resident  may  receive  the  appropriate  quality  care  and  that 
providers  are  appropriately  reimbursed.  However,  CMS’  comments  did  not 
indicate  that  it  planned  to  implement  our  recommendations  and  reorient 
its  MDS  review  program.^  Rather,  CMS’  comments  suggested  that  its 
current  efforts  provide  adequate  oversight  of  state  activities  and 
complement  state  efforts. 

While  CMS  stated  that  it  currently  evaluates,  assesses,  and  monitors  the 
accuracy  of  the  MDS  through  the  nursing  home  survey  process,  it  also 
acknowledged  the  wide  variation  in  the  adequacy  of  current  state 
accuracy  review  efforts.  Our  work  in  the  10  states  with  separate  MDS 
review  programs  raised  serious  questions  about  the  thoroughness  and 
adequacy  of  the  nursing  home  survey  process  for  reviewing  MDS 
accuracy.  Officials  in  many  of  these  states  said  that  the  survey  process 
itself  does  not  detect  MDS  accuracy  issues  as  effectively  as  separate  MDS 
review  programs.  Surveyors,  we  were  told,  do  not  have  time  to  thoroughly 


^®CMS  refers  to  the  contractor  responsible  for  this  program  as  the  DAVE  contractor. 
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review  MDS  accuracy  and  their  focus  is  on  quality  of  care  and  resident 
outcomes,  not  accuracy  of  MDS  data. 

In  response  to  our  recommendations  on  assessing  and  monitoring  the 
adequacy  of  each  state’s  MDS  reviews,  CMS  commented  that  it  would 
consider  adding  a  new  standard  to  the  state  performance  expectations  that 
the  agency  initiated  in  October  2000.  CMS  indicated  that  the  state  agency 
performance  review  program  would  result  in  a  more  comprehensive 
assessment  of  state  activities  related  to  MDS  accuracy  than  could  be 
obtained  through  the  comparative  survey  process.  CMS  also  outlined 
planned  analytic  activities — ^such  as  a  review  of  existing  state  and  private 
sector  MDS  review  methodologies  and  instruments,  ongoing 
communications  with  states  to  share  the  knowledge  gained,  and 
comprehensive  analyses  of  MDS  data  to  identify  systemic  accuracy 
problems  within  states  as  weU  as  across  states — ^that  it  believes  will  help 
to  evaluate  state  performance. 

We  agree  that  some  of  CMS’  proposed  analytic  activities  could  provide 
useful  feedback  to  states  on  problem  areas  at  the  provider,  state,  region, 
and  national  levels.  Similarly,  the  addition  of  MDS  accuracy  activities  to  its 
state  performance  standards  for  nursing  home  surveys,  which  CMS  is 
considering,  has  merit.  While  CMS  plans  to  consider  adding  a  new 
standard  to  its  state  agency  performance  review  program,  the  agency  has  a 
mechanism  in  place — ^the  comparative  survey  process — that  it  could 
readily  use  to  systematically  assess  state  performance.  However,  CMS 
apparently  does  not  intend  to  do  so.  Based  on  our  discussions  with  agency 
officials,  it  does  not  appear  that  CMS’  approach  will  yield  a  consistent 
evaluation  of  each  state’s  performance.  We  continue  to  believe  that 
assessment  and  routine  monitoring  of  each  state’s  efforts  should  be  the 
cornerstone  of  CMS’  review  program.  As  we  previously  noted,  the  agency’s 
proposed  on-site  and  off-site  reviews  of  MDS  assessments  are  too  limited 
to  systematically  assess  MDS  accuracy  in  each  state  and  would  consume 
resources  that  could  be  devoted  to  complementing  and  overseeing 
ongoing  state  activities.  A  comprehensive  review  of  the  adequacy  of  state 
MDS  accuracy  activities,  particularly  in  those  states  without  a  separate 
review  program,  is  essential  to  establish  a  baseline  and  to  allow  CMS  to 
more  efficiently  target  additional  guidance,  training,  or  technical 
assistance  that  it  acknowledged  is  necessary. 

CMS  did  not  agree  with  our  recommendation  that  it  should  provide 
guidance  to  states  regarding  adequate  documentation  in  the  medical 
record  for  each  MDS  assessment.  CMS  stated  that  requiring 
documentation  of  all  MDS  items  places  an  unnecessary  burden  on 
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facilities.  Skilled  reviewers,  it  stated,  should  be  able  to  assess  the  accuracy 
of  completed  MDS  assessments  through  a  combination  of  medical  record 
review,  observation,  and  interviews.  CMS  further  stated  that  requiring 
duplicative  documentation  might  result  in  documentation  that  is 
manufactured  and  of  questionable  accuracy.  Of  course,  the  potential  for 
manufactured  data  could  also  be  an  issue  with  the  MDS,  when  supporting 
documentation  is  absent  or  limited.  Without  adequate  documentation,  it  is 
unclear  whether  the  nursing  home  staff  sufficiently  observed  the  resident 
to  determine  his  or  her  care  needs  or  merely  checked  off  a  box  on  the 
assessment  form.  We  continue  to  believe,  as  do  most  of  the  states  with 
separate  MDS  review  programs,  that  requiring  documentation  for  the  full 
MDS  assessment  is  necessary  to  ensure  the  accuracy  of  MDS  data.  In  our 
view,  however,  this  documentation  need  not  be  duplicative  of  that  which 
is  already  in  the  medical  record  but  rather  demonstrative  of  the  basis  for 
the  higher-level  summary  judgments  about  a  resident’s  condition.  Some 
states  have  already  developed  tools  to  accomplish  this  and  in  commenting 
on  a  draft  of  this  report,  two  states  said  that  CMS  should  establish 
documentation  requirements  for  responses  on  the  MDS.  In  addition,  the 
discrepancies  cited  by  the  HHS  OIG  in  its  studies  stemmed  from 
inconsistencies  between  MDS  assessments  and  documentation  in 
residents’  medical  records.  The  OIG  acknowledged  that  the  results  of  its 
analyses  were  limited  by  the  information  available  in  the  medical  record — 
for  example,  when  a  facility  MDS  assessment  was  based  on  resident 
observation,  the  facility  may  not  have  documented  these  observations  in 
the  medical  record.  The  importance  of  adequate  documentation  is  further 
reinforced  by  the  fact  that  using  interviews  and  observation  to  validate 
MDS  assessments  may  often  not  be  possible,  particularly  for  residents 
who  have  been  discharged  from  the  nursing  home  before  an  MDS 
accuracy  review.  Given  the  importance  of  MDS  data  in  ac^justing  nursing 
home  payments  and  guiding  resident  care,  documenting  the  basis  for  the 
MDS  assessment — ^in  a  way  that  can  be  independently  validated — ^is 
critical  to  achieving  its  intended  purposes. 

CMS  provided  additional  clarifying  information  that  we  incorporated  as 
appropriate.  In  addition,  the  states  that  commented  on  the  draft  report 
generally  concurred  with  our  findings  and  provided  technical  comments 
that  we  incorporated  as  appropriate. 
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As  agreed  with  your  offices,  unless  you  publicly  announce  the  contents  of 
this  report  earlier,  we  will  not  distribute  it  until  30  days  after  its  date.  At 
that  time,  we  will  send  copies  to  the  administrator  of  CMS;  appropriate 
congressional  committees;  and  other  interested  parties.  We  will  also  make 
copies  available  to  others  upon  request. 

If  you  or  your  staff  have  any  questions,  please  call  me  at  (202)  512-7114  or 
Walter  Ochinko  at  (202)  512-7157.  Major  contributors  to  this  report  include 
Carol  Carter,  Laura  Sutton  Elsberg,  Leslie  Gordon,  and  Sandra  Gove. 

Kathryn  G.  Allen 
Director,  Health  Care — Medicaid 
and  Private  Insurance  Issues 
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Appendix  I:  Summary  of  State  On-Site  MDS 
Reviews  As  of  January  2001 


state' 

Number  of 
nursing 
horned 

Year  state  began: 

'  MDS-based 
payment  system 
•  MDS  reviews 

Review 
combined 
with  nursing 
home 
surveys? 

Survey 

findings 

used  in 

planning 

MDS 

reviews? 

Reviews 
done  on¬ 
site,  off¬ 
site,  or 
both? 

Frequency 
of  on-site 
reviews 
(all  facilities 
unless 
otherwise 
noted  Y 

Number  of 

MDS 

assessments 
reviewed  at 
each  facility 

Average 
time  lapse 
between 
facility  MDS 
and  state 
review 

lA 

465 

2000  (payment) 
2000  (reviews) 

No 

No 

Both 

Annually 

At  least  25 
percent  with  a 
minimum  of  5 
residents 

90  days 

IN 

562 

1998  (payment) 

1 998  (reviews) 

No 

No 

On-site 

only 

At  least  every 
15  months 

40  percent— -or 
no  less  than  25 
residents 

State  reviews 
most  recent 
MDS 

assessment 

ME 

126 

1993  (payment) 

1994  (reviews) 

No 

No" 

Both 

Quarterly 

Minimum 
of  10 

assessments 
per  facility 

76  days 

MS 

191 

1988  (payment) 
1992  (reviews) 

No 

No 

On-site 

only 

Annually 

At  least  20 
percent  of 
residents  in 
facility 

45  days 

OH 

1,009 

1993  (payment) 

1994  (reviews) 

No 

Not  usually* 

Both 

Annually* 

Ranging  from 
all  to  50 
residents, 
based  on 
facility  size 

State  reviews 
most  recent 
MDS 

assessment 
for  the 
reporting 
quarter 
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Appendix  I:  Summary  of  State  On-Site  MDS 

Reviews  As  of  January  2001 

Reported 
importance  of 
interviews/ 
observations 
versus  medical 
record  review^ 

State  definition 
of  “error” 

Facility  error 
rate 

calculated 

Examples  of 
remedies  and  efforts 
to  recoup  Medicaid 
payments* 

Accuracy  and  other 
trends 

Other  features  of  on¬ 
site  reviews 

Interviews  and 
observations  are 
less  important 
than  medical 
record  review 

MDS  element  not 
supported  by 
record, 

observation,  or 
interview 

Yes 

Make  referrals  to  state 
survey  agency; 
conduct  additional 
reviews;  provide  on¬ 
site  education 

During  the  first  2 
quarters  of  reviews, 
error  rate  decreased 
from  32  percent  to  22 
percent 

State  provides  voluntary 
training  sessions  on 
completing  and 
submitting  MDS 
assessments.  State 
officials  noted  that 
provider  education  is  a 
strong  focus  of  their 

MDS  review  program. 

Interviews  and 
observations  are 
equally  important 
as  medical  record 
review 

Assessment 
caused  resident  to 
be  placed  in  the 
wrong  case-mix 
category’ 

Yes 

Impose  financial 
penalties  by  reducing 
the  administrative 
component  of  a 
facility’s  Medicaid 
payment;  facility  must 
submit  plan  and  is 
subject  to  revisit; 
recalculate  case-mix 
category  and  Medicaid 
rates 

State  officials  link 
decreases  in  MDS  error 
rates  to  the  presence  of 
on-site  reviewers  and 
the  education  of 
providers 

State  publishes  annual 
guidelines  for  providers 
on  documentation 
needed  to  support  MDS 
data 

Interviews  and 
observations  are 
equally  important 
as  medical  record 
review 

MDS  element  not 
supported  by 
record, 

observation,  or 
interview” 

Yes” 

Conduct  more  frequent 
reviews;  impose 
financial  penalties;” 
request  MDS 
reassessment  from 
facility 

While  problems 
continue  In  some  MDS 
elements,  others  show 
improvement,  such  as 
ADLs 

Reviewers  bring  portable 
computers  to  facilities 
and,  using  state- 
designed  software, 
review  MDS  data 

Interviews  and 
observations  are 
equally  important 
as  medical  record 
review 

Assessment 
caused  resident  to 
be  placed  in  the 
wrong  case-mix 
category 

No 

Revisit  facilities  where 
problems  have  been 
identified;  recalculate 
case-mix  category  and 
Medicaid  rates 

Facilities  with  poor 

MDS  reviews  tend  to 
receive  many  survey 
deficiencies 

State  published 
guidelines  for  providers 
on  documentation 
needed  to  support  MDS 
data 

Interviews  and 
observations  are 
less  Important 
than  medical 
record  review 

Assessment 
caused  resident  to 
be  placed  in  the 
wrong  case-mix 
category 

Yes 

Revisit  facilities  where 
problems  have  been 
identified;  recalculate 
case-mix  category  and 
Medicaid  rates 

When  recalculating  the 
case-mix,  the  adjusted 
payments  decreased 
about  99  percent  of  the 
time 

State  has  done  the 
following  to  address 

MDS  errors:  training; 

Web  site;  MDS 
newsletter;  and  providing 
results  of  MDS  reviews 
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Appendix  I:  Summary  of  State  On-Site  MDS 

Reviews  As  of  January  2001 

state* 

Number  of 
nursing 
horned 

Year  state  began: 

•  MDS-based 
payment  system 

•  MDS  reviews 

Review 
combined 
with  nursing 
home 
surveys? 

Survey 

findings 

used  in 

planning 

MDS 

reviews? 

Reviews 
done 
on-site, 
off-site, 
or  both? 

Frequency 

of  on-site 

reviews 

(all  facilities 

unless 

otherwise 

noted)® 

Number  of 
MDS 

assessments 
reviewed  at 
each  facility 

Average 
time  lapse 
between 
facility  MDS 
and  state 
review 

PA 

774 

1 996  (payment) 

1994  (reviews) 

No 

No 

On-site 

only 

Annually 

1 5  randomly 
selected 
residents  from 
assessments 
actually  used 
in  the  rate- 
setting 
process 

6-12  months 

SD 

113 

1993  (payment) 

1993  (reviews) 

No 

Not 

usually' 

On-site 

only 

Every  15 
months 

At  least  25 
percent  of 
residents  in 
facility 

14-30  days 

VT 

43 

1 992  (payment) 

1 992  (reviews) 

No,  but  same 
staff  conduct 
reviews  and 
surveys 

Not 

usually' 

On-site 

only 

At  least 
annually 

10  percent 
pre¬ 
determined 
and/or  random 
sample  of  all 
residents  in  all 
units 

MDS  never 
older  than 

90  days 

WA 

271 

1998  (payment) 
1998  (reviews) 

No,  but  staff 
participate  in 
surveys 
about  6  times 
per  year 

Yes 

Both 

Annually 
(Staff  also 
conduct 
quarterly 
quality  review 
audits) 

Approximately 
20  percent, 
depending  on 
facility  size 

45-60  days 

Page  36 


GAO-02-279  Nursing  Home  Resident  Assessment  Data 


Appendix  I:  Summary  of  State  On-Site  MDS 

Reviews  As  of  January  2001 

Reported 
Importance  of 
interviews/ 
observations 
versus  medical 
record  review** 

State 

definition  of 
error 

Facility 
error  rate 
calculated 

Examples  of  remedies 
and  efforts  to  recoup 
Medicaid  payments* 

Accuracy  and  other  trends 

Other  features  of 
on-site  reviews 

Interviews  and 
observations  are 
less  important 
than  medical 
record  review 

Positive  MDS 
element  not 
supported  by 
record** 

Yes 

Conduct  more  frequent 
reviews;  provide  training 
within  1  month;  require 
corrective  action  plan 

State  officials  expect  that  their 
new  MDS  review  process  will 
ultimately  lead  to  a  decrease 
in  error  rates 

By  restructuring  the 
MDS  review  process, 
facilities  are  reviewed 
more  frequently, 
issues  are  identified 
more  quickly  and 
training  is  provided 
almost  immediately  to 
nursing  facility  staff 

Interviews  and 
observations  are 
equally  important 
as  medical  record 
review 

Assessment 
caused 
resident  to  be 
placed  in  the 
wrong  case- 
mix  category 

Yes 

Revisit  facilities  where 
problems  have  been 
identified;  recalculate 
case-mix  category  and 
Medicaid  rates 

Since  the  state  has  been 
reviewing  MDS  data,  the  error 
rate  has  decreased  from  about 
85  percent  to  10  percent 

On-site  reviews  also 
include  independent 
assessments  and 
inter-rater  reliability 
checks 

Interviews  and 
observations  are 
equally  important 
as  medical  record 
review 

MDS  element 
not  supported 
by  record, 
observation, 
or  interview 
(effective 
10/1/01) 

Yes 

(effective 

10/1/01) 

Impose  financial 
penalties  (none  imposed 
to  date);  revisit  facilities 
where  problems  have 
been  identified; 
recalculate  case-mix 
category  and  Medicaid 
rates 

State  officials  told  us  that 
Vermont  facilities  do  not  have 
serious  MDS  accuracy  issues 

Vermont  tried  to 
combine  MDS 
reviews  with  nursing 
home  surveys,  but 
found  that  It 
detracted  from  the 
survey  process 

Interviews  and 
observations  are 
equally  important 
as  medical  record 
review 

Assessment 
caused 
resident  to  be 
placed  in  the 
wrong  case- 
mix  category 

Yes 

Impose  financial 
penalties  (none  imposed 
to  date);  revisit  facilities 
where  problems  have 
been  identified; 
recalculate  case-mix 
category  and  Medicaid 
rates 

The  types  of  MDS  errors  that 
commonly  reoccur  relate  to 
misapplication  of  MDS 
definitions,  and  may  in  large 
part  be  due  to  facility  staff 
turnover.  In  commenting  on  a 
draft  of  this  report,  officials  told 
us  that  these  errors  are 
consistent  with  those  found  in 
other  states  with  MDS-based 
payment  systems. 

State  plans  to  publish 
the  results  of  MDS 
accuracy  reviews  on 
a  Web  page  to 
prevent  simple  but 
recurring  errors 

“Virginia  is  not  included  because  of  the  newness  of  its  MDS  review  program  (began  operating  in  April 
2001).  We  have  included  the  nine  other  states  with  longer  standing  on-site  review  programs, 

‘’Source:  CMS  Nursing  Home  Compare  Web  site,  http://www.medicare.gov/nhcompare/Search, 
printed  6/8/01. 

"This  column  reflects  the  frequency  of  initial  reviews  for  each  facility.  Some  states  conduct  follow-up 
reviews  more  frequently  for  facilities  where  problems  have  been  identified. 
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Appendix  I:  Summary  of  State  On-Site  MDS 
Reviews  As  of  January  2001 


"We  asked  states  to  select  from  the  following  categories:  more  important,  equally  important,  and  less 
important. 

“In  addition,  ail  nine  states  reported  that  they  refer  cases  of  suspected  fraud  to  their  state’s  Medicaid 
Fraud  Control  Unit. 

'Indiana  officials  added  the  following  language  to  characterize  MDS  errors:  An  error  occurs  when  the 
audit  findings  are  different  from  the  facility’s  transmitted  MDS  data  and  those  differences  result  in  a 
different  case-mix  category. 

“Survey  findings  may  be  used  to  plan  MDS  reviews,  although  this  has  not  occurred  yet. 

''Financial  penalties  and  facility  error  rates,  however,  are  only  based  on  errors  that  result  in  changes 
for  a  subset  of  case-mix  categories. 

‘Survey  findings  are  occasionally  used  in  planning  MDS  reviews. 

'Staff  use  risk  analysis  to  select  approximately  200  facilities  per  year  for  on-site  reviews. 

“Pennsylvania  reviews  only  those  MDS  elements  that  have  a  positive  response.  For  example,  if  a 
facility  responded  “no”  or  left  an  MDS  element  blank,  that  item  would  not  be  reviewed  for  accuracy, 
even  if  it  could  affect  the  case-mix  category  for  that  particular  resident. 
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Appendix  II:  Comments  from  the  Centers  for 
Medicare  and  Medicaid  Services 


DEPARTMENT  OF  HEALTH  &  HUMAN  SERVICES 


Centers  for  Medicare  &  Medicaid  Services 


Administrator 
Washington,  DC  20201 


DATE: 


JAN  3  0  2002 


TO: 


FROM: 


SUBJECT: 


Kathryn  G.  Allen 
Associate  Director 
Health  Care — Medicaid  and  Private 
Insurance  Issues 


Thomas  A.  Scully 
Administrator 


General  Accounting  Office  Draft  Report,  pursing  Homes:  Federal  Efforts 
to  Monitor  Resident  Assessment  Data  Should  Complement  State  Activities 
(GAa-02-279) 


Thank  you  for  the  opportunity  to  review  and  comment  on  the  above-referenced  report 
regarding  Federal  and  state  efforts  to  monitor  resident  assessment  data  and  to  ensure  the 
accuracy  of  the  minimum  data  set  (MDS).  The  Centers  for  Medicare  &  Medicaid 
Services  (CMS)  recognizes  the  coding  of  items  on  the  MDS  impacts  reimbursement  and 
care  planning.  It  is  essential  that  the  assessment  data  reflect  the  resident’s  health  status, 
so  that  the  resident  may  receive  the  appropriate  quality  care  and  that  providers  are 
reimbursed  appropriately. 

When  automation  requirements  were  implemented  in  1998,  CMS  devoted  significant 
resources  to  the  development  of  an  accuracy  improvement  program.  We  instructed  a 
contractor  to  develop  MDS  accuracy  review  protocols.  Then  we  funded  a  program 
safeguard  contractor,  known  as  the  Data  Assessment  and  Verification  (DAVE) 
contractor,  to  audit  and  verify  MDS  data. 

The  CMS  also  developed  and  implemented  a  major  MDS  system  enhancement  that 
provided  new  mechanisms  to  correct  inaccurate  information  residing  in  the  MDS 
database.  Accuracy  was  significantly  improved  with  the  addition  of  this  system  (e.g., 
approximately  66  percent  reduction  in  the  proportion  of  records  in  the  database 
containing  invalid  data  values). 

We  currently  evaluate,  assess,  and  monitor  the  accuracy  of  the  MDS  through  the  nursing 
home  survey  process.  According  to  Task  5C  of  Appendix  P  Survey  Procedures  for  Long- 
Term  Care,  ‘‘after  observing  and  talking  with  the  resident,  the  surveyor  conducts  a 
comprehensive  review  which  includes  the  following:  a  check  of  specific  items  on  the 
MDS  for  accurate  coding  for  the  resident's  condition-  The  specific  items  to  be  checked 
will  be  based  on  the  Quality  Indicators  (Qls)  identified  for  the  resident  on  the  Resident 
Level  Summary. 
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Appendix  II;  Conunents  from  the  Centers  for 
Medicare  and  Medicaid  Services 


Page  2  -  Kathryn  G.  Allen 

At  least  2  of  the  QIs  identified  for  the  resident  must  be  matched  against  the  QI  definitions 
and  against  evidence  other  than  the  MDS  to  verity  that  the  resident’s  condition  is 
accurately  recorded  in  the  MDS.  Keep  in  mind  that  you  are  verifying  that  the  resident’s 
condition  was  accurately  assessed  at  the  time  the  MDS  was  completed.” 


We  appreciate  the  effort  that  went  into  this  report  and  the  opportunity  to  review  and 
comment  on  the  issues  it  raises.  Our  comments  on  the  GAO  recommendations  follow. 
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Appendix  II:  Comments  from  the  Centers  for 
Medicare  and  Medicaid  Services 


Page  3  -  Katfiryn  G.  AJlcn 
GAO  Recommendation 

With  the  goal  of  complementing  and  leveraging  the  considerable  Federal,  State, 
nursing  home  resources  already  devoted  to  nursing  home  surveys  and  to  s^arate  MDS 
accuracy  review  programs,  we  recommend  that  the  Administrator  of  CMS: 

•  Review  file  adequacy  of  current  state  efforts  to  ensure  the  «ccur»ey«f^®S 

data,  and  provide,  where  necessary,  additionai  guidance,  training  or  technical 
assistance. 

(;!MS  Response 

We  agree  that  assessing  the  adequacy  of  state  efforts  to  ensure  the  accuracy  of  MDS  data 
is  an  important  oversight  function.  Development  of  analytic  tools  to  monitor  and 
compare  State  activities  is  included  in  the  DAVE  scope  of  work.  The  CMS  considered 
alternatives  to  MDS  and  Outcome  and  Assessment  Information  Set  (OASIS)  accurwy 
verification  before  deciding  on  the  more  centralized  focus  of  the  DAVE  contract  The 
CMS  thinks  that  this  national  approach  to  accuracy  is  better  positioned  to  impact 
accuracy  across  all  states,  recognizing  that  current  state  efforts  varies  wdely  in  adequacy 
and  reflects  different  special  interests  within  states. 

The  DAVE  contract  includes  many  tasks  that  will  evaluate  state  pcifonnance  related  to 
MDS  accuracy  and  the  provision  of  training  and  technical  assistance.  Dunng  the  cariy 
phases  of  the  DAVE  contract,  the  contractor  will  review  existing  data  depcnden^ls 
and  instruments  (e.g.,  state  agency  and  private  sector  entities)  used  to  momtor  MDS  data 
accuracy.  This  assessment  will  include  how  methodologies  used  in  existing  systems  can 
be  blended  into  their  review  efforts.  Further,  CMS  and  tfic  DAVE  contractor  yrill  have 

ongoing  communications  with  the  state  agency  communities  to  discuss  activities 

necessary  to  support  data  assessment  and  verification  efforts  and  to  share  the  knowledge 
gained. 

The  DAVE  reports  can  be  developed  to  identify  systemic  accur^  pn^lems  vrithin  states 
fadlitics  with  consistently  high  numbers  of  residents  classifying  into  clinically 
conmlex  solely  due  to  the  number  of  lAysician  orders,  high  num^rs  of  patirats  lecdving 
ultra  high  therapy,  etc),  as  well  as  across  states.  Establishing  nation^  baseline  thresholds 
for  MDS  and  OASIS  data  and  iq>plicable  associated  claims  will  furnish  the  national, 
state,  and  provider  level  evidence  wc  need  to  address  areas  of  concern  for  CMS:  program 
inte^ty;  beneficiary  health  and  safety;  imd  quality  improvement  While  some  states  md 
fiscal  intermediaries  (FIs)  are  already  doing  a  limited  level  of  this  analysis,  they  lack  the 
data  and  staff  resources  to  do  an  ongoing  comprehensive  analysis.  The  DAVE  contractor 
can  give  the  state  agencies  and  FIs  analytic  files.  Ihe  states  and  Fk  can  provide 
feedback  on  major  problem  areas  at  provider,  state,  region,  and  national  levels.  The 
DAVE  on-site  reviews  can  then  complement  the  state  efforts. 

The  CMS  understands  the  need  for  continual  state  and  provider  MDS  training  to  improve 
the  accuracy  of  MDS  assessments.  The  CMS  funded  an  accuracy  protocol 
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development  contract.  In  August  1999,  we  analyzed  the  findings  of  this  contract  and 
published  two  sets  of  questions  and  answers,  released  in  March  2001  and  July  2001  on 
CMS’s  Web  site.  The  questions  and  answers  address  the  areas  of  concern  identified  in 
“Figure  1  ”  of  the  GAO  report.  (Note  the  reference  standard  data  are  firom  facilities 
located  in  states  with  existing  auditing  systems,  (Pennsylvania,  Washington,  Ohio).  This 
infbnnation  was  also  used  as  the  basis  for  the  development  of  a  special  MDS  2.0  training 
session  for  state  MDS  coordinators  that  was  provided  during  the  July  2001  MDS 
conference.  The  same  information  will  be  used  to  guide  future  revisions  of  the  MDS 
instrument. 


GAO  Recommendation 

•  Monitor  the  adequacy  of  state  MOS  accuracy  activities  on  an  ongoing  basis,  such 
as  through  the  use  of  the  established  federal  comparative  survey  process. 

CMS  Response 

We  agree  that  state  agency  training  and  oversight  functions  are  crucial  to  ensuring 
accuracy  of  MDS  data.  We  believe  that,  under  the  DAVE  contract,  we  will  be  able  to 
significantly  upgrade  our  capabilities  to  monitor  state  ^ency  activity.  The  DAVE  scope 
of  work  includes  reports  that  CMS  can  use  to  evaluate  MDS  accuracy  on  state,  regional, 
and  national  levels.  These  reports  will  provide  CMS  with  the  baseline  data  needed  to 
analyze  provider  data  by  state  and  region.  Once  baselines  are  established,  we  can  use 
statistical  analysis  to  highlight  aberrant  coding  patterns  that  impact  quality  and 
reimbursement.  We  plan  to  communicate  this  infoimation  to  state  agencies  so  they  can 
better  focus  their  training  efforts.  At  the  same  time,  we  will  be  able  to  use  the  DAVE 
reports  to  review  the  effectiveness  of  state  training  and  oversight  activities. 

The  CMS  monitors  state  survey  agencies’  performance  in  several  areas  to  ensure  that 
standards  are  met  and  to  identify  any  necessary  corrective  action.  Each  year  we  identify 
the  performance  areas  to  be  included  in  the  evaluation.  Since  CMS  funds  states  in  order 
to  provide  training  and  technical  assistance  to  nursing  homes,  we  will  consider  this  area 
for  inclusion  in  future  state  survey  agency  performance  review  protocols.  We  are 
confident  that  the  state  agency  performance  review  program  will  result  in  a  more 
comprdhensive  assessment  of  state  activities  related  to  MDS  accuracy  (through  training 
and  technical  assistance)  than  could  be  obtained  through  the  comparative  survey  process. 
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GAO  Recommendation 

•  Provide  guidance  to  state  agencies  and  nursing  homes  that  sufficient  evidentiary 
documentation  to  support  the  full  MDS  assessment  be  included  In  residents* 
medical  records. 

CMS  Response 

We  do  not  agree  that  duplicative  documentation  of  MDS  items  is  necessary  or  desirable. 
The  MDS,  as  a  clinical  assessment,  is  an  integral  part  of  the  resident’s  record.  The 
CMS’s  position  is  that  additional  documentation  for  all  MDS  items  creates  unnecessary 
burden  for  facilities.  There  are  however,  by  exception,  just  a  few  MDS  items  for  which  a 
second  source  of  documentation  is  required. 

The  MDS,  in  conjunction  with  other  clinical  documentation,  provides  a  full  view  of  the 
beneficiaiy’s  clinical  course  in  a  given  time  period.  Validation  of  MDS  responses 
generally  requires  a  review  of  information  fiom  medical  records  and  other  sources.  In 
evaluating  assessments,  the  reviewer  must  be  able  to  exercise  clinical  judgment  in 
determining  the  plausibility  of  MDS  responses  in  light  of  other  information  in  the 
resident’s  medical  record.  If  we  were  to  require  duplicative  documentation  to  support 
comparative  reviews,  that  documentation  may  be  manufactured  for  the  sole  purpose  of 
satisfying  a  comparative  audit,  and  may  be  of  questionable  accuracy.  Skilled 
reviewers/auditors  are  able  to  assess  the  accuracy  of  completed  MDS  through  a 
combination  of  reviews,  the  comprehensive  record,  observations,  and  interviews. 
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GAO’S  Mission 

■ 

The  General  Accounting  Office,  the  investigative  arm  of  Congress,  exists  to 
support  Congress  in  meeting  its  constitutional  responsibilities  and  to  help 
improve  the  performance  and  accountability  of  the  federal  government  for  the 
American  people.  GAO  examines  the  use  of  public  funds;  evaluates  federal 
programs  and  policies;  and  provides  analyses,  recommendations,  and  other 
assistance  to  help  Congress  make  informed  oversight,  policy,  and  funding 
decisions.  GAO’s  commitment  to  good  government  is  reflected  in  its  core  values 
of  accountability,  integrity,  and  reliability. 

Obtaining  Copies  of 
GAO  Reports  and 
Testimony 

The  fastest  and  easiest  way  to  obtain  copies  of  GAO  documents  is  through  the 
Internet.  GAO’s  Web  site  (www.gao.gov)  contains  abstracts  and  full-text  files  of 
current  reports  and  testimony  and  an  expanding  archive  of  older  products.  The 

Web  site  features  a  search  engine  to  help  you  locate  documents  using  key  words 
and  phrases.  You  can  print  these  documents  in  their  entirety,  including  charts  and 
other  graphics. 

Each  day,  GAO  issues  a  list  of  newly  released  reports,  testimony,  and 
correspondence.  GAO  posts  this  list,  known  as  “Today’s  Reports,”  on  its  Web  site 
daily.  The  list  contains  links  to  the  full-text  document  files.  To  have  GAO  e-mail 
this  list  to  you  every  afternoon,  go  to  www.gao.gov  and  select  "Subscribe  to  daily 
e-mail  alert  for  newly  released  products"  under  the  GAO  Reports  heading. 

Order  by  Mail  or  Phone 

The  first  copy  of  each  printed  report  is  free.  Additional  copies  are  $2  each,  A 
check  or  money  order  should  be  made  out  to  the  Superintendent  of  Documents. 
GAO  also  accepts  VISA  and  Mastercard.  Orders  for  100  or  more  copies  mailed  to  a 
single  address  are  discounted  25  percent.  Orders  should  be  sent  to: 

U.S.  General  Accounting  Office 

P.O.  Box  37050 

Washington,  D.C.  20013 

To  order  by  Phone:  Voice:  (202)  512-6000 

TDD:  (202)  512-2537 

Fax:  (202)  512-6061 

Visit  GAO’s  Document 
Distribution  Center 

GAO  Building 

Room  1100,  700  4th  Street,  NW  (comer  of  4th  and  G  Streets,  NW) 

Washington,  D.C.  20013 

To  Report  Fraud, 
Waste,  and  Abuse  in 
Federal  Programs 

■ - -  ' 

Contact: 

Web  site:  www.gao.gov/fraudnet/fraudnet.htm, 

E-mail:  fraudnet@gao.gov,  or 

1-800424-5454  or  (202)  512-7470  (automated  answering  system). 

Public  Affairs 

Jeff  NeUigan,  Managing  Director,  NeliiganJ@gao,gov  (202)  512-4800 

U.S.  General  Accounting  Office,  441  G.  Street  NW,  Room  7149, 

Washington,  D.C.  20548 
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